Python's magic numbers

I've been troubleshooting a problem that arose while running a pyc file: RuntimeError: Bad magic number in .pyc file

OK, so how do I find the magic number that is in the myfile.pyc and how do I find the magic number that was expected?

The magic number is the first two bytes of the file. I can get them using xxd:

robert@dante ~/MathScripts
$ xxd myfile.pyc | head -n 1
0000000: 330d 0d0a 33c7 925b 4c29 0000 e300 0000  3...3..[L)......

In other words, the magic number of the python36 version of myfile.pyc is 330d 0d0a.

The python27 version has a different magic number:

$ xxd ../MathScripts_py27/myfile.pyc | head -n 1
0000000: 03f3 0d0a 24d4 f75a 6300 0000 0000 0000  ....$..Zc.......

…namely 03f3 0d0a.

Having got the magic number of my byte-code file, how do I discover the magic number that my current python is expecting? The commands are different in python2 and 3. In python2:

robert@dante ~
$ source activate py27
(py27)
robert@dante ~
$ python
Python 2.7.15 |Anaconda, Inc.| (default, May  1 2018, 23:32:55)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import imp
>>> imp.get_magic().encode('hex')
'03f30d0a'

And in python3:

robert@dante ~
$ source activate py36
(py36)
robert@dante ~
$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 26 2018, 08:42:37)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import imp
>>> imp.get_magic().hex()
'330d0d0a'

There's a full list of magic numbers here.

More on pass

The approach to password management that I am trialing (detailed here) is almost perfect. The one thing that was bugging me was that I could not get bash completion to work. And when I am visiting http://this-site-has-a-weirdly-longURL.com that becomes something of a nuisance.

What was the problem? Short version: a missing forward slash in a directory name.

Long version: I had installed the password store in a Dropbox subfolder so that I could access it on multiple machines. That meant that I needed to set the environment variable PASSWORD_STORE_DIR to its location. Consequently I had this line in ~/.bash_profile:

export PASSWORD_STORE_DIR=~/Dropbox/.password-store

This looked like it was working. pass was storing and recalling passwords quite happily; the password store was synchronizing across my machines. So why the heck was bash completion not working?

Next step: try bash completion after I have turned on command and parameter logging. I do this in bash thus:

$ set -x

The effect of this command is

$ help set
(snip)
-x  Print commands and their arguments as they are executed

When I have pass attempt to complete amazon.com after the first two characters, I get this:

$ pass am+ COMPREPLY=()
+ local cur=am
+ local 'commands=init ls find grep show insert generate edit rm mv cp git help version'
+ [[ 1 -gt 1 ]]
+ COMPREPLY+=($(compgen -W "${commands}" -- ${cur}))
++ compgen -W 'init ls find grep show insert generate edit rm mv cp git help version' -- am
+ _pass_complete_entries 1
+ prefix=/Users/robert/Dropbox/.password-store
+ suffix=.gpg
+ autoexpand=1
+ local 'IFS=
'
+ items=($(compgen -f $prefix$cur))
++ compgen -f /Users/robert/Dropbox/.password-stoream
+ local items

That compgen command in the penultimate line does not look correct, does it? It rather looks as if I need to add a terminating / to the value in PASSWORD_STORE_DIR.

So I turn off logging (set +x), append the forward-slash to the directory name and bingo, bash completion is working.