Skip to content

Conversation

bastelfreak
Copy link
Contributor

We have some weird JRuby issues on main. Previously we pinned to JRuby-9.4 on Ubuntu-24.04. That currently gives us https://github.com/ruby/ruby-builder/releases/download/toolcache/jruby-9.4.13.0-ubuntu-24.04.tar.gz

The old Perforce CI was pinned to jruby-9.4.3.0 on Ubuntu 22.04, lets try that.

@bastelfreak
Copy link
Contributor Author

9.4.3:

Finished in 13 minutes 14 seconds
24007 examples, 15 failures, 50 pending

9.4.8:

Finished in 13 minutes 21 seconds
24007 examples, 39 failures, 33 pending

9.4.13:

Finished in 13 minutes 13 seconds
24007 examples, 39 failures, 33 pending

@bastelfreak
Copy link
Contributor Author

nest run with more jrubies:

9.4.3:

Finished in 13 minutes 18 seconds
24007 examples, 15 failures, 50 pending

9.4.4:

Finished in 13 minutes 14 seconds
24007 examples, 16 failures, 49 pending

9.4.5:

Finished in 14 minutes 27 seconds
24007 examples, 16 failures, 49 pending

9.4.6:

Finished in 12 minutes 53 seconds
24007 examples, 39 failures, 33 pending

9.4.7:

Finished in 13 minutes 4 seconds
24007 examples, 39 failures, 33 pending

9.4.8:

Finished in 15 minutes 12 seconds
24007 examples, 39 failures, 33 pending

9.4.13

Finished in 13 minutes 32 seconds
24007 examples, 39 failures, 33 pending

@bastelfreak
Copy link
Contributor Author

Perforce used bundle exec rake parallel:spec[2]. Lets see if there are some race conditions and switch from bundle exec rake parallel:spec to bundle exec rake parallel:spec[2].

@bastelfreak
Copy link
Contributor Author

ok with bundle exec rake parallel:spec[2]

9.4.3:

Finished in 12 minutes 34 seconds
24007 examples, 15 failures, 50 pending

9.4.4:

Finished in 12 minutes 27 seconds
24007 examples, 16 failures, 49 pending

9.4.5:

Finished in 12 minutes 22 seconds
24007 examples, 16 failures, 49 pending

9.4.6:

Finished in 12 minutes 43 seconds
24007 examples, 39 failures, 33 pending

9.4.7:

Finished in 12 minutes 44 seconds
24007 examples, 39 failures, 33 pending

9.4.8:

Finished in 12 minutes 55 seconds
24007 examples, 39 failures, 33 pending

9.4.13:

Finished in 12 minutes 53 seconds
24007 examples, 39 failures, 33 pending

@bastelfreak bastelfreak force-pushed the jruby branch 2 times, most recently from 9febbef to c1461c5 Compare June 19, 2025 12:10
@bastelfreak bastelfreak mentioned this pull request Jun 19, 2025
We have some weird JRuby issues on main. Previously we pinned to
JRuby-9.4 on Ubuntu-24.04. That currently gives us
https://github.com/ruby/ruby-builder/releases/download/toolcache/jruby-9.4.13.0-ubuntu-24.04.tar.gz

The old Perforce CI was pinned to jruby-9.4.3.0 on Ubuntu 22.04, lets
try that.
bastelfreak pushed a commit that referenced this pull request Aug 24, 2025
A bit of backstory. I did a lot of JRuby debugging in #122. Yesterday @sebastianrakel and I did a bit of deep diving. Some numbers:

* puppetserver 8.7.0, the last open source release from Perforce, ships JRuby 9.4.8.0
* openvox-server 8.9.0 ships JRuby 9.4.8.0
* puppet enterprise 2025.4.0 (latest version at 2025-07-27) ships pe-puppetserver JRuby 9.4.8.0
* latest JRuby 9.4 is 9.4.13.0
* JRuby 9.4 series is compatible with MRI Ruby 3.1
* puppetserver 7 uses JRuby 9.3 which is compatible with Ruby 2.6
* Perforce never tested JRuby 9.4.8.0 in CI, JRuby was pinned to 9.4.3.0

Since openvox runs tests, the Rspec JRuby pipeline fails. We cannot access the old perforce pipeline logs. Their last successful CI run I could fine was https://github.com/puppetlabs/puppet/actions/runs/11693993097. This was triggered by commit puppetlabs/puppet@4ea6ee2 on 2024-11-06.

In https://github.com/puppetlabs/puppet/actions/runs/12021455668 we see the first failed pipelline for puppetserver 8, triggered by  puppetlabs/puppet@e967115 on 2024-11-24. The used JRuby verison in both jobs was 9.4.3.0

Something changed between 2024-11-06 and 2024-11-24 and I doubt it was related to puppetlabs/puppet@e967115 (puppetlabs/puppet#9523).

Whatever changed in that timeframe modified the error reporting. Basic example:

```
class a {}
class a {}
```

raises:

```
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Class 'a' is already defined (file: /etc/puppetlabs/code/environments/server/manifests/site.pp, line: 1); cannot redefine (file: /etc/puppetlabs/code/environments/server/manifests/site.pp, line: 2) (file: /etc/puppetlabs/code/environments/server/manifests/site.pp, line: 2) on node puppettestinfra.voxpupu.li
```

After "cannot redefine", the file is listed twice. This issue is reproducible puppetserver 8.7.0, Puppet Enterprise 2025.4.0 & openvox-server 8.9.0.

After some testing in #122 I noticed that JRuby 9.4.3.0 indeed raises errors now, but others. I assume the JRuby issues are, at least in part, triggered by a webmock update. I don't know yet what causes the duplicate file lines in the output. The old rspec tests demand a single file line in the output.

In #164 I'm testing different Jruby Versions with a webmock pinned to 3.25.0. 3.25.0 was released on 2025-02-06. The version before that, 3.24.0, on 2024-09-30. That's what Perforce was using back then.

And after diffing CI output from JRuby 9.4.3, 9.4.4, 9.4.5 & 9.4.6 we can see that the duplicate line messages start with JRuby 9.4.6: https://gist.github.com/bastelfreak/025c23417f9dd7215b5c5af205fc8b76
@bastelfreak bastelfreak added the skip-changelog Excluded from CHANGELOG label Aug 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip-changelog Excluded from CHANGELOG
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant