Segmentation faults on RISC-V in `pthread_create` for modules other than SoftHSM

[hertelukas]

We are using cryptoki on RISC-V and encounter segmentation faults when using any pkcs11 module other than SoftHSM (which does work and passes all tests).

For our own investigation, we created this short example:

use std::env;

use cryptoki::context::{CInitializeArgs, Pkcs11};

fn main() {
    let args: Vec<String> = env::args().collect();
    let pkcs11 = Pkcs11::new(args.get(1).unwrap()).expect("failed to load pkcs11 module");
    println!("Module loaded");
    pkcs11
        .initialize(CInitializeArgs::OsThreads)
        .expect("failed to initialize pkcs11");
    println!("PKCS11 initialized");
    let slot = pkcs11.get_slots_with_token().expect("could not get slot")[0];
    println!("slot chosen");
    let session = pkcs11.open_ro_session(slot).unwrap();
    println!("Here are some random bytes: {:?}", session.generate_random_vec(8));
}

When using OpenSCs module opensc-pkcs11.so, we get the following backtrace with gdb, indicating a failure at Pkcs11::new(...):

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7809502 in pthread_create () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff7809502 in pthread_create () from /usr/lib/libc.so.6
#1  0x00007ffff742c4cc in ?? () from /usr/lib/libglib-2.0.so.0
#2  0x00007ffff742d6e2 in g_thread_pool_new_full () from /usr/lib/libglib-2.0.so.0
#3  0x00007ffff7617d68 in ?? () from /usr/lib/libgio-2.0.so.0
#4  0x00007ffff7617eba in g_task_get_type () from /usr/lib/libgio-2.0.so.0
#5  0x00007ffff7687b1a in ?? () from /usr/lib/libgio-2.0.so.0
#6  0x00007ffff7679ff8 in g_bus_get_sync () from /usr/lib/libgio-2.0.so.0
#7  0x00007ffff764b20c in g_application_register () from /usr/lib/libgio-2.0.so.0
#8  0x00007ffff7fc6702 in ?? () from /usr/lib/opensc-pkcs11.so
#9  0x000000000008f8ce in call_init ()
#10 0x000000000008f99a in _dl_init ()
#11 0x00000000000729e0 in _dl_catch_exception ()
#12 0x0000000000073bca in dl_open_worker ()
#13 0x00000000000729bc in _dl_catch_exception ()
#14 0x0000000000073e40 in _dl_open ()
#15 0x0000000000067102 in dlopen_doit ()
#16 0x00000000000729bc in _dl_catch_exception ()
#17 0x0000000000072a0e in _dl_catch_error ()
#18 0x0000000000066fce in _dlerror_run ()
#19 0x000000000006717c in dlopen ()
#20 0x000000000001c06a in libloading::os::unix::{impl#2}::open::{closure#0}<&std::path::Path> () at /home/lukas/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libloading-0.8.9/src/os/unix/mod.rs:190
#21 0x000000000001be64 in libloading::os::unix::with_dlerror<libloading::os::unix::Library, libloading::os::unix::{impl#2}::open::{closure_env#0}<&std::path::Path>, libloading::error::Error> (closure=<error reading variable: Cannot access memory at address 0x1000>, error=0x1b8e8 <core::ops::function::FnOnce::call_once<libloading::os::unix::{impl#2}::open::{closure_env#1}<&std::path::Path>, (&core::ffi::c_str::CStr)>>)
    at /home/lukas/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libloading-0.8.9/src/os/unix/mod.rs:69
#22 0x000000000001bf50 in libloading::os::unix::Library::open<&std::path::Path> (filename=..., flags=1) at /home/lukas/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libloading-0.8.9/src/os/unix/mod.rs:188
#23 0x000000000001bef4 in libloading::os::unix::Library::new<&std::path::Path> (filename=...) at /home/lukas/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libloading-0.8.9/src/os/unix/mod.rs:135
#24 0x000000000001c2d4 in libloading::safe::Library::new<&std::path::Path> (filename=...) at /home/lukas/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libloading-0.8.9/src/safe.rs:85
#25 0x000000000001b74a in cryptoki_sys::Pkcs11::new<&std::path::Path> (path=...) at /home/lukas/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/cryptoki-sys-0.4.0/src/bindings/riscv64gc-unknown-linux-gnu.rs:5941
#26 0x000000000001bd16 in cryptoki::context::Pkcs11::new<&alloc::string::String> (filename=0x122fc8) at /home/lukas/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/cryptoki-0.10.0/src/context/mod.rs:118
#27 0x0000000000017616 in pkcs11_test::main () at src/main.rs:7

For another HSM, the Thales SafeNet eToken, using even another module, we get a similar error, however a bit later, during pkcs11.initialize(...):

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff75fa502 in pthread_create () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff75fa502 in pthread_create () from /usr/lib/libc.so.6
#1  0x00007ffff7f83bb8 in C_Initialize () from ./safenet/this/outLinux/debug32/libeToken.so
#2  0x0000000000025bfc in cryptoki::context::general_purpose::initialize (ctx=0x7ffffffff530) at src/context/general_purpose.rs:19
#3  cryptoki::context::Pkcs11::initialize (self=0x7ffffffff530) at src/context/mod.rs:195
#4  0x000000000001768e in pkcs11_test::main () at src/main.rs:10

Both those modules do work on x86 with the same example, and also with the pkcs11-tool on RISC-V.

[hug-dev]

Hello 👋

Thanks for the detailed bug report! I can propose two paths to debug this further:

• does multithreading work at all on your RISC-V platform? You could try creating a small program creating multiple threads (making sure it would use pthread_create) and checking that it works?
• you can use, as a small "hack", the CKF_LIBRARY_CANT_CREATE_OS_THREADS option when initializing the library as opposed as the current CInitializeArgs::OsThreads. For that you would need to add the new variant here and its bindings.