View Issue Details

IDProjectCategoryView StatusLast Update
0007372CentOS-7CentOS-7-Pluspublic2015-04-07 19:29
Reportergreno2 
PrioritynormalSeverityminorReproducibilityN/A
Status resolvedResolutionfixed 
Product Version7.0-1406 
Target VersionFixed in Version 
Summary0007372: Enable ceph-Module into centosplus-Kernel
DescriptionHi,

please provide ceph as part of centosplus-Kernel (or as separate kmod) as this currently only works with elrepo's kernel-ml under centos7 and kernel-ml is "far away" from rhel7-kernel/default.
TagsNo tags attached.
abrt_hash
URL

Activities

toracat

toracat

2014-07-14 16:06

manager   ~0020428

Will try to enable it in the next kernel update.

< # CONFIG_CEPH_LIB is not set
---
> CONFIG_CEPH_LIB=m
> # CONFIG_CEPH_LIB_PRETTYDEBUG is not set
> # CONFIG_CEPH_LIB_USE_DNS_RESOLVER is not set

< # CONFIG_CEPH_FS is not set
---
> CONFIG_CEPH_FS=m
neufeind

neufeind

2014-07-19 11:07

reporter   ~0020472

I see there was a release some day after your posting, but still without ceph. Did that overlap somehow or did you encouter problems during the build?
toracat

toracat

2014-07-19 12:52

manager   ~0020473

That (3.10.0-123.4.2.el7.centos.plus.0.1) was an "emergency" update to take care of a security issue. All the planned changes for the plus kernel will be applied when the distro kernel gets updated. My test build with the current kernel version did not show any problem with ceph enabled.
toracat

toracat

2014-07-31 11:53

manager   ~0020556

The latest kernel-plus (3.10.0-123.4.4.el7.centos.plus) has the requested feature enabled.
toracat

toracat

2015-01-05 20:40

manager   ~0022076

* Important Note about a possible security issue *

There is a known vulnerability related to Ceph:

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-6416

"Buffer overflow in net/ceph/auth_x.c in Ceph, as used in the Linux kernel before 3.16.3, allows remote attackers to cause a denial of service (memory corruption and panic) or possibly have unspecified other impact via a long unencrypted auth ticket. "

According to this upstream (RH) bugzilla:

https://bugzilla.redhat.com/show_bug.cgi?id=1142060

"This issue did not affect the versions of the Linux kernel packages as shipped with Red Hat Enterprise Linux 5, 6, 7 and Red Hat Enterprise MRG 2 (as they did not include support for this feature)."

However, the issue likely affects the centosplus kernel now that Ceph is enabled. There is a patch available from kernel.org:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c27a3e4d667fdcad3db7b104f75659478e0c68d8

Problem is that the patch cannot be applied cleanly to the RH/CentOS kernel source code. Therefore, unless we come up with a way to mitigate the issue, we will have to disable Ceph in the next kernel update.

Additional note: the upcoming EL 7.1 kernel will [most probably] have Ceph enabled.
toracat

toracat

2015-01-11 18:31

manager   ~0022104

Last edited: 2015-01-11 18:37

View 2 revisions

Ceph source code based on kernel-3.10.0-123.9.1.el7 is available from:

https://github.com/ceph/ceph-client/commit/ee04310a6245bb4fd0940cfa6a1e785b97aac4e7#diff-e8ee0538aa9326745487b1a162690885

It contains the security fix quoted above and works with the current CentOS kernel. We will use this code in the next kernel update.

[patch uploaded]

toracat

toracat

2015-01-11 18:36

manager  

centos-linux-3.10-libceph-CVE-2014-6416-bug7372.patch (9,603 bytes)
centosplus patch bug#7372

Security issue: 
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-6416

Patch:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c27a3e4d667fdcad3db7b104f75659478e0c68d8

Solution:
The patch from kernel.org does not cleanly apply in the RHEL source code.
However, ceph changes for kernel-3.10.0-123.9.1.el7 available from
github work and include the patch.

https://github.com/ceph/ceph-client/commit/ee04310a6245bb4fd0940cfa6a1e785b97aac4e7#diff-e8ee0538aa9326745487b1a162690885

Provided-by: Ken Dreyer <ktdreyer@ktdreyer.com>
Applied-by: Akemi Yagi <toracat@centos.org>

--- a/net/ceph/auth_x.c	2014-12-12 16:14:08.000000000 -0800
+++ b/net/ceph/auth_x.c	2015-01-10 10:35:30.037735402 -0800
@@ -13,8 +13,6 @@
 #include "auth_x.h"
 #include "auth_x_protocol.h"
 
-#define TEMP_TICKET_BUF_LEN	256
-
 static void ceph_x_validate_tickets(struct ceph_auth_client *ac, int *pneed);
 
 static int ceph_x_is_authenticated(struct ceph_auth_client *ac)
@@ -64,7 +62,7 @@ static int ceph_x_encrypt(struct ceph_cr
 }
 
 static int ceph_x_decrypt(struct ceph_crypto_key *secret,
-			  void **p, void *end, void *obuf, size_t olen)
+			  void **p, void *end, void **obuf, size_t olen)
 {
 	struct ceph_x_encrypt_header head;
 	size_t head_len = sizeof(head);
@@ -75,8 +73,14 @@ static int ceph_x_decrypt(struct ceph_cr
 		return -EINVAL;
 
 	dout("ceph_x_decrypt len %d\n", len);
-	ret = ceph_decrypt2(secret, &head, &head_len, obuf, &olen,
-			    *p, len);
+	if (*obuf == NULL) {
+		*obuf = kmalloc(len, GFP_NOFS);
+		if (!*obuf)
+			return -ENOMEM;
+		olen = len;
+	}
+
+	ret = ceph_decrypt2(secret, &head, &head_len, *obuf, &olen, *p, len);
 	if (ret)
 		return ret;
 	if (head.struct_v != 1 || le64_to_cpu(head.magic) != CEPHX_ENC_MAGIC)
@@ -129,139 +133,115 @@ static void remove_ticket_handler(struct
 	kfree(th);
 }
 
-static int ceph_x_proc_ticket_reply(struct ceph_auth_client *ac,
-				    struct ceph_crypto_key *secret,
-				    void *buf, void *end)
+static int process_one_ticket(struct ceph_auth_client *ac,
+			      struct ceph_crypto_key *secret,
+			      void **p, void *end)
 {
 	struct ceph_x_info *xi = ac->private;
-	int num;
-	void *p = buf;
+	int type;
+	u8 tkt_struct_v, blob_struct_v;
+	struct ceph_x_ticket_handler *th;
+	void *dbuf = NULL;
+	void *dp, *dend;
+	int dlen;
+	char is_enc;
+	struct timespec validity;
+	struct ceph_crypto_key old_key;
+	void *ticket_buf = NULL;
+	void *tp, *tpend;
+	void **ptp;
+	struct ceph_timespec new_validity;
+	struct ceph_crypto_key new_session_key;
+	struct ceph_buffer *new_ticket_blob;
+	unsigned long new_expires, new_renew_after;
+	u64 new_secret_id;
 	int ret;
-	char *dbuf;
-	char *ticket_buf;
-	u8 reply_struct_v;
 
-	dbuf = kmalloc(TEMP_TICKET_BUF_LEN, GFP_NOFS);
-	if (!dbuf)
-		return -ENOMEM;
+	ceph_decode_need(p, end, sizeof(u32) + 1, bad);
 
-	ret = -ENOMEM;
-	ticket_buf = kmalloc(TEMP_TICKET_BUF_LEN, GFP_NOFS);
-	if (!ticket_buf)
-		goto out_dbuf;
+	type = ceph_decode_32(p);
+	dout(" ticket type %d %s\n", type, ceph_entity_type_name(type));
 
-	ceph_decode_need(&p, end, 1 + sizeof(u32), bad);
-	reply_struct_v = ceph_decode_8(&p);
-	if (reply_struct_v != 1)
+	tkt_struct_v = ceph_decode_8(p);
+	if (tkt_struct_v != 1)
 		goto bad;
-	num = ceph_decode_32(&p);
-	dout("%d tickets\n", num);
-	while (num--) {
-		int type;
-		u8 tkt_struct_v, blob_struct_v;
-		struct ceph_x_ticket_handler *th;
-		void *dp, *dend;
-		int dlen;
-		char is_enc;
-		struct timespec validity;
-		struct ceph_crypto_key old_key;
-		void *tp, *tpend;
-		struct ceph_timespec new_validity;
-		struct ceph_crypto_key new_session_key;
-		struct ceph_buffer *new_ticket_blob;
-		unsigned long new_expires, new_renew_after;
-		u64 new_secret_id;
-
-		ceph_decode_need(&p, end, sizeof(u32) + 1, bad);
-
-		type = ceph_decode_32(&p);
-		dout(" ticket type %d %s\n", type, ceph_entity_type_name(type));
-
-		tkt_struct_v = ceph_decode_8(&p);
-		if (tkt_struct_v != 1)
-			goto bad;
 
-		th = get_ticket_handler(ac, type);
-		if (IS_ERR(th)) {
-			ret = PTR_ERR(th);
-			goto out;
-		}
+	th = get_ticket_handler(ac, type);
+	if (IS_ERR(th)) {
+		ret = PTR_ERR(th);
+		goto out;
+	}
 
-		/* blob for me */
-		dlen = ceph_x_decrypt(secret, &p, end, dbuf,
-				      TEMP_TICKET_BUF_LEN);
-		if (dlen <= 0) {
-			ret = dlen;
-			goto out;
-		}
-		dout(" decrypted %d bytes\n", dlen);
-		dend = dbuf + dlen;
-		dp = dbuf;
-
-		tkt_struct_v = ceph_decode_8(&dp);
-		if (tkt_struct_v != 1)
-			goto bad;
+	/* blob for me */
+	dlen = ceph_x_decrypt(secret, p, end, &dbuf, 0);
+	if (dlen <= 0) {
+		ret = dlen;
+		goto out;
+	}
+	dout(" decrypted %d bytes\n", dlen);
+	dp = dbuf;
+	dend = dp + dlen;
 
-		memcpy(&old_key, &th->session_key, sizeof(old_key));
-		ret = ceph_crypto_key_decode(&new_session_key, &dp, dend);
-		if (ret)
-			goto out;
+	tkt_struct_v = ceph_decode_8(&dp);
+	if (tkt_struct_v != 1)
+		goto bad;
 
-		ceph_decode_copy(&dp, &new_validity, sizeof(new_validity));
-		ceph_decode_timespec(&validity, &new_validity);
-		new_expires = get_seconds() + validity.tv_sec;
-		new_renew_after = new_expires - (validity.tv_sec / 4);
-		dout(" expires=%lu renew_after=%lu\n", new_expires,
-		     new_renew_after);
+	memcpy(&old_key, &th->session_key, sizeof(old_key));
+	ret = ceph_crypto_key_decode(&new_session_key, &dp, dend);
+	if (ret)
+		goto out;
 
-		/* ticket blob for service */
-		ceph_decode_8_safe(&p, end, is_enc, bad);
-		tp = ticket_buf;
-		if (is_enc) {
-			/* encrypted */
-			dout(" encrypted ticket\n");
-			dlen = ceph_x_decrypt(&old_key, &p, end, ticket_buf,
-					      TEMP_TICKET_BUF_LEN);
-			if (dlen < 0) {
-				ret = dlen;
-				goto out;
-			}
-			dlen = ceph_decode_32(&tp);
-		} else {
-			/* unencrypted */
-			ceph_decode_32_safe(&p, end, dlen, bad);
-			ceph_decode_need(&p, end, dlen, bad);
-			ceph_decode_copy(&p, ticket_buf, dlen);
-		}
-		tpend = tp + dlen;
-		dout(" ticket blob is %d bytes\n", dlen);
-		ceph_decode_need(&tp, tpend, 1 + sizeof(u64), bad);
-		blob_struct_v = ceph_decode_8(&tp);
-		new_secret_id = ceph_decode_64(&tp);
-		ret = ceph_decode_buffer(&new_ticket_blob, &tp, tpend);
-		if (ret)
+	ceph_decode_copy(&dp, &new_validity, sizeof(new_validity));
+	ceph_decode_timespec(&validity, &new_validity);
+	new_expires = get_seconds() + validity.tv_sec;
+	new_renew_after = new_expires - (validity.tv_sec / 4);
+	dout(" expires=%lu renew_after=%lu\n", new_expires,
+	     new_renew_after);
+
+	/* ticket blob for service */
+	ceph_decode_8_safe(p, end, is_enc, bad);
+	if (is_enc) {
+		/* encrypted */
+		dout(" encrypted ticket\n");
+		dlen = ceph_x_decrypt(&old_key, p, end, &ticket_buf, 0);
+		if (dlen < 0) {
+			ret = dlen;
 			goto out;
+		}
+		tp = ticket_buf;
+		ptp = &tp;
+		tpend = *ptp + dlen;
+	} else {
+		/* unencrypted */
+		ptp = p;
+		tpend = end;
+	}
+	ceph_decode_32_safe(ptp, tpend, dlen, bad);
+	dout(" ticket blob is %d bytes\n", dlen);
+	ceph_decode_need(ptp, tpend, 1 + sizeof(u64), bad);
+	blob_struct_v = ceph_decode_8(ptp);
+	new_secret_id = ceph_decode_64(ptp);
+	ret = ceph_decode_buffer(&new_ticket_blob, ptp, tpend);
+	if (ret)
+		goto out;
 
-		/* all is well, update our ticket */
-		ceph_crypto_key_destroy(&th->session_key);
-		if (th->ticket_blob)
-			ceph_buffer_put(th->ticket_blob);
-		th->session_key = new_session_key;
-		th->ticket_blob = new_ticket_blob;
-		th->validity = new_validity;
-		th->secret_id = new_secret_id;
-		th->expires = new_expires;
-		th->renew_after = new_renew_after;
-		dout(" got ticket service %d (%s) secret_id %lld len %d\n",
-		     type, ceph_entity_type_name(type), th->secret_id,
-		     (int)th->ticket_blob->vec.iov_len);
-		xi->have_keys |= th->service;
-	}
+	/* all is well, update our ticket */
+	ceph_crypto_key_destroy(&th->session_key);
+	if (th->ticket_blob)
+		ceph_buffer_put(th->ticket_blob);
+	th->session_key = new_session_key;
+	th->ticket_blob = new_ticket_blob;
+	th->validity = new_validity;
+	th->secret_id = new_secret_id;
+	th->expires = new_expires;
+	th->renew_after = new_renew_after;
+	dout(" got ticket service %d (%s) secret_id %lld len %d\n",
+	     type, ceph_entity_type_name(type), th->secret_id,
+	     (int)th->ticket_blob->vec.iov_len);
+	xi->have_keys |= th->service;
 
-	ret = 0;
 out:
 	kfree(ticket_buf);
-out_dbuf:
 	kfree(dbuf);
 	return ret;
 
@@ -270,6 +250,34 @@ bad:
 	goto out;
 }
 
+static int ceph_x_proc_ticket_reply(struct ceph_auth_client *ac,
+				    struct ceph_crypto_key *secret,
+				    void *buf, void *end)
+{
+	void *p = buf;
+	u8 reply_struct_v;
+	u32 num;
+	int ret;
+
+	ceph_decode_8_safe(&p, end, reply_struct_v, bad);
+	if (reply_struct_v != 1)
+		return -EINVAL;
+
+	ceph_decode_32_safe(&p, end, num, bad);
+	dout("%d tickets\n", num);
+
+	while (num--) {
+		ret = process_one_ticket(ac, secret, &p, end);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+
+bad:
+	return -EINVAL;
+}
+
 static int ceph_x_build_authorizer(struct ceph_auth_client *ac,
 				   struct ceph_x_ticket_handler *th,
 				   struct ceph_x_authorizer *au)
@@ -583,13 +591,14 @@ static int ceph_x_verify_authorizer_repl
 	struct ceph_x_ticket_handler *th;
 	int ret = 0;
 	struct ceph_x_authorize_reply reply;
+	void *preply = &reply;
 	void *p = au->reply_buf;
 	void *end = p + sizeof(au->reply_buf);
 
 	th = get_ticket_handler(ac, au->service);
 	if (IS_ERR(th))
 		return PTR_ERR(th);
-	ret = ceph_x_decrypt(&th->session_key, &p, end, &reply, sizeof(reply));
+	ret = ceph_x_decrypt(&th->session_key, &p, end, &preply, sizeof(reply));
 	if (ret < 0)
 		return ret;
 	if (ret != sizeof(reply))
@@ -709,3 +718,4 @@ out:
 }
 
 
+
toracat

toracat

2015-01-12 21:31

manager   ~0022113

A patched version of kernel-plus (3.10.0-123.13.2.el7.bug7372.centos.plus) is available for testing:

http://people.centos.org/toracat/kernel/7/plus/bug7372/

Please note that these packages are not official, not signed. They are provided for testing purposes.
sdainard

sdainard

2015-01-12 23:57

reporter   ~0022114

Can you also set the Ceph block device flag?

CONFIG_BLK_DEV_RBD=m
toracat

toracat

2015-01-13 00:41

manager   ~0022115

Last edited: 2015-01-13 01:56

View 2 revisions

Will do.

[Edit: note added]

config BLK_DEV_RBD
tristate "Rados block device (RBD)"
depends on INET && BLOCK
select CEPH_LIB
select LIBCRC32C
select CRYPTO_AES
select CRYPTO
default n
help
  Say Y here if you want include the Rados block device, which stripes
  a block device over objects stored in the Ceph distributed object
  store.

  More information at http://ceph.newdream.net/.

toracat

toracat

2015-01-29 01:34

manager   ~0022250

kernel-3.10.0-123.20.1.el7 is out.
Changing the config to CONFIG_BLK_DEV_RBD=m resulted in a compilation error. Therefore it was not included in this update. Pending investigation.
toracat

toracat

2015-01-29 01:36

manager   ~0022251

The patch in the testing release is now in kernel-plus-3.10.0-123.20.1.el7.centos.plus.
visbits

visbits

2015-02-28 05:32

reporter   ~0022421

Easily fixed and compiles successfully.

Edit Lines of "drivers/block/rbd.c":
#2174 struct bio *bio_list;
#2176 struct page **pages;

#2174 struct bio *bio_list = NULL;
#2176 struct page **pages = NULL;
toracat

toracat

2015-02-28 06:15

manager  

rdb-bug7372.patch (521 bytes)
--- a/drivers/block/rbd.c	2014-12-12 16:14:08.000000000 -0800
+++ b/drivers/block/rbd.c	2015-02-27 22:09:13.630593652 -0800
@@ -2171,9 +2171,9 @@ static int rbd_img_request_fill(struct r
 	struct rbd_obj_request *obj_request = NULL;
 	struct rbd_obj_request *next_obj_request;
 	bool write_request = img_request_write_test(img_request);
-	struct bio *bio_list;
+	struct bio *bio_list = NULL;
 	unsigned int bio_offset = 0;
-	struct page **pages;
+	struct page **pages = NULL;
 	u64 img_offset;
 	u64 resid;
 	u16 opcode;
rdb-bug7372.patch (521 bytes)
toracat

toracat

2015-02-28 06:16

manager   ~0022422

Thanks. Patch file (rdb-bug7372.patch) uploaded. Will give it a shot shortly.
toracat

toracat

2015-03-01 07:55

manager   ~0022427

Confirmed. The fix will be in the next kernel update.
toracat

toracat

2015-04-07 19:29

manager   ~0022699

As of CentOS 7.1-1503, ceph module is now enabled in the distro kernel.

Issue History

Date Modified Username Field Change
2014-07-14 11:26 greno2 New Issue
2014-07-14 12:34 toracat Status new => assigned
2014-07-14 16:06 toracat Note Added: 0020428
2014-07-19 11:07 neufeind Note Added: 0020472
2014-07-19 12:52 toracat Note Added: 0020473
2014-07-31 11:53 toracat Note Added: 0020556
2015-01-05 20:40 toracat Note Added: 0022076
2015-01-11 18:31 toracat Note Added: 0022104
2015-01-11 18:36 toracat File Added: centos-linux-3.10-libceph-CVE-2014-6416-bug7372.patch
2015-01-11 18:37 toracat Note Edited: 0022104 View Revisions
2015-01-12 21:31 toracat Note Added: 0022113
2015-01-12 23:57 sdainard Note Added: 0022114
2015-01-13 00:41 toracat Note Added: 0022115
2015-01-13 01:56 toracat Note Edited: 0022115 View Revisions
2015-01-29 01:34 toracat Note Added: 0022250
2015-01-29 01:36 toracat Note Added: 0022251
2015-02-28 05:32 visbits Note Added: 0022421
2015-02-28 06:15 toracat File Added: rdb-bug7372.patch
2015-02-28 06:16 toracat Note Added: 0022422
2015-03-01 07:55 toracat Note Added: 0022427
2015-04-07 19:29 toracat Note Added: 0022699
2015-04-07 19:29 toracat Status assigned => resolved
2015-04-07 19:29 toracat Resolution open => fixed