Monday, July 16, 2018

AES CBC Encryption on OpenVMS

It turns out that using the convenience function ENCRYPT$ENCRYPT_ONE_RECORD on OpenVMS for AES CBC does something squirrelly.  It will not let you pass in an IV.  Instead, ENCRYPT$INIT sets the IV to 16 bytes of \0.  In order to ensure your identical plaintext first blocks aren't identical after encryption, your first 16 byte block needs to be your true IV.  This will get XORd with the null byte encrypted IV and sufficiently scrambled so your second block can be uniquely encrypted.  OR just do it right and avoid the convenience method.  Use ENCRYPT$INIT properly.

Ref: http://h41379.www4.hpe.com/doc/83final/4493/4493pro_025.html#encrypt_one_routine

"For AES, the optional P1 argument for the AES IV initialization vector is a reference to a 16-byte (2 quadword) value.
If you omit this argument, the initialization vector used is the residue of the previous use of the specified context block. ENCRYPT$INIT initializes the context block with an initialization vector of zero."