This might be a user error... But I had to do:
openssl req -x509 -newkey rsa:4096 -config=/fultonm/prod/ssl/openssl.cnf -keyout key.pem -out cert.pem -sha256 -days 365
instead of just
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 365
so I don't know if I built it wrong or if people have to set up their own config in practice.