Encryption on Android 5-based ROMs

(This should be a reply to @sddfdds and @ivzn on http://forum.xda-developers.com/show...postcount=3274, but this forum doesn't allow new members to post to "developer" subforums. If a moderator wants to merge the threads, please do!)

I've been trying to upgrade a Galaxy Nexus GSM (maguro) from CyanogenMod 11 to @Ziyan's CyanogenMod 12.1 builds, but before I can use CM12.1 as my daily-use ROM, I need to get device encryption to work. The symptom is that if I encrypt /data on CM11, then flash CM12.1 without wiping /data, the system fails to decrypt it during boot; or if I wipe /data, install CM12.1 (as a new installation or an upgrade), and then enable encryption, the system reports that encryption failed and recommends a factory reset.

Looking into this based on the hint from @ivzn's bisection, it seems that system/vold in Android ≥ 5.1 assumes features of the dm-crypt module that aren't supported by the 3.0-based kernel used on Galaxy Nexus. I suspect any 5.1-based ROM is going to have a similar issue: I've noticed that TWRP 2.8.7.0 also can't decrypt an encrypted /data, and I suspect this might be because it's also using a newer vold.

One way to resolve that would be to patch vold, as @ivzn suggested. The linked commit on Gerrit doesn't seem to be available any more, but the general idea seems to be something like this:

Code:

---

diff --git a/cryptfs.c b/cryptfs.c
index 880ac53..e1cdab7 100755
--- a/cryptfs.c
+++ b/cryptfs.c
@@ -1035,7 +1035,7 @@ static int load_crypto_mapping_table(struct crypt_mnt_ftr *crypt_ftr, unsigned c
 #else
  convert_key_to_hex_ascii(master_key, crypt_ftr->keysize, master_key_ascii);
 #endif
-  sprintf(crypt_params, "%s %s 0 %s 0 %s 0", crypt_ftr->crypto_type_name,
+  sprintf(crypt_params, "%s %s 0 %s 0 %s", crypt_ftr->crypto_type_name,
          master_key_ascii, real_blk_name, extra_params);
 
  SLOGI("%s: target_type = %s\n", __func__, tgt->target_type);

---

However, since vold is common to all devices, that comes with some risk of breaking unrelated devices; I can't be sure that there isn't some random device (perhaps with a modified dm-crypt for hardware-assisted crypto) where that trailing 0 is somehow important.

Another way to avoid that would be to patch the kernel to tolerate this extra "column", like it does in 3.4-based kernels. That's what I've suggested here:

http://review.cyanogenmod.org/#/c/110508/
https://github.com/smcv/android_kern...ommit/6407247d

and that does seem to work fine on my maguro. This has the advantage that it only affects the Galaxy Nexus (tuna) family of devices, so it's less likely to cause a regression.

I suspect that this is applicable to all 5.1-based ROMs for Galaxy Nexus. Similarly, if there are other devices that have a working Android-5.1-based ROM but are still stuck on a 3.0 kernel, they'd probably benefit from the same patch.

I hope this is useful to someone.