Expand ARM Architecture Compatibility #5954
Comments
|
I am totally in favor of extending ARM architecture compatibility. This will allow Java-Tron to run on more platforms and take advantage of the benefits of the ARM architecture, such as higher energy efficiency and lower cost.
|
|
It sounds great, but I am a novice in ARM architecture. I am curious about the challenges of supporting ARM architecture. Can you list something like a task list in the future? It is convenient to clearly understand the current status and future challenges. |
|
Here are some common considerations: Important
Important 2. Native code
Tip 3. Endianness Tip 4. Memory alignment: Tip 5. Atomic operations and concurrency Caution 6. Floating-point arithmetic Tip 7. Performance optimization Important 8. Third-party dependencies
Important 9. Build and deployment process:
Tip 10. Hardware feature dependencies: Tip 11. System calls and OS interactions Important 12. Cross-platform testing
|
|
@halibobo1205 Do you want to support ARM Architecture and latest JVM version at the same time ? Or just support ARM Architecture using JDK8 ? |
|
@317787106 JVM officially supports ARM:
According to Oracle Java SE Support Roadmap, JDK9 and JDK16 are non-LTS, and JDK 17 is LTS. Based on the above information, I propose that ARM support JDK17 as a minimum. Warning This is the last planned update of JDK 17 under the NFTC. Updates after September 2024 will be licensed under the Java SE OTN License (OTN) and production use beyond the limited free grants of the OTN license will require a fee. |
Regarding JDK version compatibility. |
|
When is the expected completion time for this work? It sounds like a big project. |
|
@endiaoekoe I propose that ARM support JDK17 as a minimum. |
|
@abn2357 Tron currently only supports JDK 8, based on the above information, JDK17 supports ARM fully, perhaps Tron needs to upgrade JDK17 first, which is another big project. |
|
@halibobo1205 It sounds great, and I look forward to your implementation. I see that there may be subtle differences in floating point precision and behavior between ARM and x86. When supporting it, be sure to ensure data consistency. Also investigate whether there are other places that may cause data inconsistency. |
|
This issue has been added to the core devs community call #22, welcome to share the latest progress @halibobo1205, and discuss together with @endiaoekoe @tomatoishealthy @317787106 @zeusoo001 @abn2357. |
|
1. JDK version compatibility
|
|
6. Floating-point arithmetic known issues:
Unfortunately, Tron does use private long exchangeToSupply(long balance, long quant) {
logger.debug("balance: " + balance);
long newBalance = balance + quant;
logger.debug("balance + quant: " + newBalance);
double issuedSupply = -supply * (1.0 - Math.pow(1.0 + (double) quant / newBalance, 0.0005));
logger.debug("issuedSupply: " + issuedSupply);
long out = (long) issuedSupply;
supply += out;
return out;
}
private long exchangeFromSupply(long balance, long supplyQuant) {
supply -= supplyQuant;
double exchangeBalance =
balance * (Math.pow(1.0 + (double) supplyQuant / supply, 2000.0) - 1.0);
logger.debug("exchangeBalance: " + exchangeBalance);
return (long) exchangeBalance;
}Test case @Test
public void testPow() {
double x = 29218;
double q = 4761432;
double ret = Math.pow(1.0 + x / q, 0.0005);
double ret2 = StrictMath.pow(1.0 + x / q, 0.0005);
System.out.printf("%s%n", doubleToHex(ret)); // 3ff000033518c576
System.out.printf("%s%n", doubleToHex(ret2)); // 3ff000033518c575
Assert.assertEquals(0, Double.compare(ret, ret2)); // fail in jdk8_X86, success in jdk8_ARM64
}
public static String doubleToHex(double input) {
// Convert the starting value to the equivalent value in a long
long doubleAsLong = Double.doubleToRawLongBits(input);
// and then convert the long to a hex string
return Long.toHexString(doubleAsLong);
}Tron Should Use StrictMath to Avoid Cross-Platform Consistency Issues. To help ensure the portability on ARM for Java-Tron, I suggest a new proposal to convert Math to StrictMath. cc @zeusoo001 |
|
@halibobo1205 First support JDK8 on mac ARM and then extend to support JDK17 on linux and mac ARM may be smooth. |
Does this mean that there is no longer a dependency between ARM architecture upgrade and JDK upgrade? In addition, TRON only focuses on Oracle JDK, right? |
|
2. Native code JNI (Java Native Interface) or other native code.
|
|
8. Third-party dependencies Ensure all third-party libraries and dependencies support ARM architecture.
|
|
Warning This is the last planned update of JDK 17 under the NFTC. Updates after September 2024 will be licensed under the Java SE OTN License (OTN) and production use beyond the limited free grants of the OTN license will require a fee. To avoid subsequent charges for commercial use, I recommend switching to OpenJDK. |
|
Caution Strong data consistency and finality |
|
1. JDK version compatibility |
|
A hard fork solution will be introduced in 4.8.0, switching floating-point calculations from Math to StrictMath. |
|
Currently, java-tron supports both LevelDB and RocksDB. On the ARM architecture, we intend to support only RocksDB, mainly due to the following considerations:
This will ensure the best database usage experience on ARM architecture. |
Mac spends much time on CI test due to this: |
|
Important On the ARM architecture, we intend to support only RocksDB When using RocksDB on the CI test, we found that some tests failed due to differences in behavior between LevelDB and RocksDB.
|
|
This issue has been added to the tronprotocol/pm#105, welcome to share the latest progress @halibobo1205 , and discuss together with @endiaoekoe @317787106 @tomatoishealthy @zeusoo001 @abn2357 . |
|
When can Tron node run on ARM architecture，do you have an approximate time? |
|
@sYNCHROtime, It's expected that java-tron would be able to run on ARM platform in Q2 2025 if the corresponding dependency updation and code adaption goes smoothly. |
|
Hi @halibobo1205 , my team is building an MPC enterprise wallet, are there any updates on this? are we still looking at Q2? |
|
@dkatzan Yes, most feature adaptations have been completed and are expected to be online in Q2. |
|
@halibobo1205 Thx, that's great news I posted my full issue here, any help would be greatly appreciated |
|
@dkatzan You can try installing the x86_64 JDK on the m1, which will then use the Rosetta emulation directly. |
|
Trace Floating-point arithmetic :
Tron uses
|
|
Trace Floating-point arithmetic :
I'll go over the first option in detail, and then the pros and cons of the 2 options Detailed Description of Hardcoded Special Cases:Option 1 uses partial hardcoding to handle special cases, with the following implementation steps:
comparison the two options:
|
|
For historical data(the Bancor trading pair), until the new proposal 101 took effect, 48 POW calculation instances caused final block state inconsistencies for Main-Net. Simulating the x86(x87 ) pow instruction requires the following work:
JDK8 pow JVM code: for more details: https://github.com/openjdk/jdk/blob/jdk8-b120/hotspot/src/cpu/x86/vm/macroAssembler_x86.cpp#L2269 void MacroAssembler::increase_precision() {
subptr(rsp, BytesPerWord);
fnstcw(Address(rsp, 0));
movl(rax, Address(rsp, 0));
orl(rax, 0x300);
push(rax);
fldcw(Address(rsp, 0));
pop(rax);
}
void MacroAssembler::restore_precision() {
fldcw(Address(rsp, 0));
addptr(rsp, BytesPerWord);
}
void MacroAssembler::pow_exp_core_encoding() {
// kills rax, rcx, rdx
subptr(rsp,sizeof(jdouble));
// computes 2^X. Stack: X ...
// f2xm1 computes 2^X-1 but only operates on -1<=X<=1. Get int(X) and
// keep it on the thread's stack to compute 2^int(X) later
// then compute 2^(X-int(X)) as (2^(X-int(X)-1+1)
// final result is obtained with: 2^X = 2^int(X) * 2^(X-int(X))
fld_s(0); // Stack: X X ...
frndint(); // Stack: int(X) X ...
fsuba(1); // Stack: int(X) X-int(X) ...
fistp_s(Address(rsp,0)); // move int(X) as integer to thread's stack. Stack: X-int(X) ...
f2xm1(); // Stack: 2^(X-int(X))-1 ...
fld1(); // Stack: 1 2^(X-int(X))-1 ...
faddp(1); // Stack: 2^(X-int(X))
// computes 2^(int(X)): add exponent bias (1023) to int(X), then
// shift int(X)+1023 to exponent position.
// Exponent is limited to 11 bits if int(X)+1023 does not fit in 11
// bits, set result to NaN. 0x000 and 0x7FF are reserved exponent
// values so detect them and set result to NaN.
movl(rax,Address(rsp,0));
movl(rcx, -2048); // 11 bit mask and valid NaN binary encoding
addl(rax, 1023);
movl(rdx,rax);
shll(rax,20);
// Check that 0 < int(X)+1023 < 2047. Otherwise set rax to NaN.
addl(rdx,1);
// Check that 1 < int(X)+1023+1 < 2048
// in 3 steps:
// 1- (int(X)+1023+1)&-2048 == 0 => 0 <= int(X)+1023+1 < 2048
// 2- (int(X)+1023+1)&-2048 != 0
// 3- (int(X)+1023+1)&-2048 != 1
// Do 2- first because addl just updated the flags.
cmov32(Assembler::equal,rax,rcx);
cmpl(rdx,1);
cmov32(Assembler::equal,rax,rcx);
testl(rdx,rcx);
cmov32(Assembler::notEqual,rax,rcx);
movl(Address(rsp,4),rax);
movl(Address(rsp,0),0);
fmul_d(Address(rsp,0)); // Stack: 2^X ...
addptr(rsp,sizeof(jdouble));
}
void MacroAssembler::fast_pow() {
// computes X^Y = 2^(Y * log2(X))
// if fast computation is not possible, result is NaN. Requires
// fallback from user of this macro.
// increase precision for intermediate steps of the computation
increase_precision();
fyl2x(); // Stack: (Y*log2(X)) ...
pow_exp_core_encoding(); // Stack: exp(X) ...
restore_precision();
} |
Principle of X87 Instruction Simulation
Simulation implementation(c++)
Float80 ln_coe[] = {
Float80(0x3FFF, 0x8000000000000000), // 1.0
Float80(0x3FFD, 0xFFFFFFFFFFFFFFFF), // 0.5
Float80(0x3FFD, 0xAAAAAAAAAAAAAAAA), // 0.333...
Float80(0x3FFC, 0xFFFFFFFFFFFFFFFF), // 0.25
Float80(0x3FFC, 0xCCCCCCCCCCCCCCCC), // 0.2
Float80(0x3FFC, 0xAAAAAAAAAAAAAAAA), // 0.166...
};
Float80 taylor_ln2_float80(Float80 x) {
return x -
x * x * ln_coe[1] +
x * x * x * ln_coe[2] -
x * x * x * x * ln_coe[3] +
x * x * x * x * x * ln_coe[4] -
x * x * x * x * x * x * ln_coe[5];
}
// ln(2)^n/n!
Float80 exp_coe[] = {
Float80(0.69314718055994530942),
Float80(0.24022650695910071233),
Float80(0.05550410866482157995),
Float80(0.00961812910762847716),
Float80(0.00133335581464284434),
};
Float80 taylor_exp2_float80(Float80 x) {
return x * exp_coe[0] +
x * x * exp_coe[1] +
x * x * x * exp_coe[2] +
x * x * x * x * exp_coe[3] +
x * x * x * x * x * exp_coe[4];
}
double taylor_pow2_float80(double x, double y) {
Float80 x80(x), y80(y);
Float80 ln2(0x3FFE, 0xB17217F7D1CF7BBB);
Float80 y_lg2_x = y80 * taylor_ln2_float80(x80 - Float80(1)) / ln2;
// For the test dataset, since y_lg2_x<1, this step omits the exponentiation of the integer part
Float80 exp_y_lg2_x = taylor_exp2_float80(y_lg2_x) + Float80(1);
return exp_y_lg2_x.to_double();
}
Data("3ff0192278704be3", 0.0005, "3ff000033518c576"); // 4137160
Data("3ff000002fc6a33f", 0.0005, "3ff0000000061d86"); // 4065476
Data("3ff00314b1e73ecf", 0.0005, "3ff0000064ea3ef8"); // 4071538
Data("3ff0068cd52978ae", 0.0005, "3ff00000d676966c"); // 4109544
Data("3ff0032fda05447d", 0.0005, "3ff0000068636fe0"); // 4123826
Data("3ff00051c09cc796", 0.0005, "3ff000000a76c20e"); // 4166806
Data("3ff00bef8115b65d", 0.0005, "3ff0000186893de0"); // 4225778
Data("3ff009b0b2616930", 0.0005, "3ff000013d27849e"); // 4251796
Data("3ff00364ba163146", 0.0005, "3ff000006f26a9dc"); // 4257157
Data("3ff019be4095d6ae", 0.0005, "3ff0000348e9f02a"); // 4260583
Data("3ff0123e52985644", 0.0005, "3ff0000254797fd0"); // 4367125
Data("3ff0126d052860e2", 0.0005, "3ff000025a6cde26"); // 4402197
Data("3ff0001632cccf1b", 0.0005, "3ff0000002d76406"); // 4405788
Data("3ff0000965922b01", 0.0005, "3ff000000133e966"); // 4490332
Data("3ff00005c7692d61", 0.0005, "3ff0000000bd5d34"); // 4499056
Data("3ff015cba20ec276", 0.0005, "3ff00002c84cef0e"); // 4518035
Data("3ff00002f453d343", 0.0005, "3ff000000060cf4e"); // 4533215
Data("3ff006ea73f88946", 0.0005, "3ff00000e26d4ea2"); // 4647814
Data("3ff00a3632db72be", 0.0005, "3ff000014e3382a6"); // 4766695
Data("3ff000c0e8df0274", 0.0005, "3ff0000018b0aeb2"); // 4771494
Data("3ff00015c8f06afe", 0.0005, "3ff0000002c9d73e"); // 4793587
Data("3ff00068def18101", 0.0005, "3ff000000d6c3cac"); // 4801947
Data("3ff01349f3ac164b", 0.0005, "3ff000027693328a"); // 4916843
Data("3ff00e86a7859088", 0.0005, "3ff00001db256a52"); // 4924111
Data("3ff00000c2a51ab7", 0.0005, "3ff000000018ea20"); // 5098864
Data("3ff020fb74e9f170", 0.0005, "3ff00004346fbfa2"); // 5133963
Data("3ff00001ce277ce7", 0.0005, "3ff00000003b27dc"); // 5139389
Data("3ff005468a327822", 0.0005, "3ff00000acc20750"); // 5151258
Data("3ff00006666f30ff", 0.0005, "3ff0000000d1b80e"); // 5185021
Data("3ff000045a0b2035", 0.0005, "3ff00000008e98e6"); // 5295829
Data("3ff00e00380e10d7", 0.0005, "3ff00001c9ff83c8"); // 5380897
Data("3ff00c15de2b0d5e", 0.0005, "3ff000018b6eaab6"); // 5400886
Data("3ff00042afe6956a", 0.0005, "3ff0000008892244"); // 5864127
Data("3ff0005b7357c2d4", 0.0005, "3ff000000bb48572"); // 6167339
Data("3ff00033d5ab51c8", 0.0005, "3ff0000006a279c8"); // 6240974
Data("3ff0000046d74585", 0.0005, "3ff0000000091150"); // 6279093
Data("3ff0010403f34767", 0.0005, "3ff0000021472146"); // 6428736
Data("3ff00496fe59bc98", 0.0005, "3ff000009650a4ca"); // 6432355,6493373
Data("3ff0012e43815868", 0.0005, "3ff0000026af266e"); // 6555029
Data("3ff00021f6080e3c", 0.0005, "3ff000000458d16a"); // 7092933
Data("3ff000489c0f28bd", 0.0005, "3ff00000094b3072"); // 7112412
Data("3ff00009d3df2e9c", 0.0005, "3ff00000014207b4"); // 7675535
Data("3ff000def05fa9c8", 0.0005, "3ff000001c887cdc"); // 7860324
Data("3ff0013bca543227", 0.0005, "3ff00000286a42d2"); // 8292427
Data("3ff0021a2f14a0ee", 0.0005, "3ff0000044deb040"); // 8517311
Data("3ff0002cc166be3c", 0.0005, "3ff0000005ba841e"); // 8763101
Data("3ff0000cc84e613f", 0.0005, "3ff0000001a2da46"); // 9269124
Data("3ff000057b83c83f", 0.0005, "3ff0000000b3a640"); // 9631452
exp:0.0005 base:1.00628495434413 3ff019be4095d6ae
expected: 1.0000031326481 3ff0000348e9f02a
result: 1.0000031326481 3ff0000348e9f029
exp:0.0005 base:1.00805230779141 3ff020fb74e9f170
expected: 1.00000401003852 3ff00004346fbfa2
result: 1.00000401003852 3ff00004346fbfa1
|
|
diff data(48 POW calculation instances) is the result of Math and StrictMath, and if algorithmic simulations are performed, the implementation needs to be fully tested for full equivalence with the Math library. |
|
|
In the Java HotSpot virtual machine, Intrinsics in the HotSpot JVM are special, optimized implementations of commonly used Java methods. When the JVM identifies specific method calls, it may replace the standard Java implementation of these methods with more efficient native code. Specifically for the
This optimization mechanism is usually managed by the do_intrinsic(_dlog, java_lang_Math, log_name, double_double_signature, F_S)Through this intrinsic function optimization, the HotSpot JVM allows Java programs to maintain platform independence while achieving performance close to native code, which is particularly beneficial for math-computation-intensive applications. Here's the logic for |
|
@halibobo1205 When Math calculates pow(double,double), how can you determine if the result is inconsistent with that calculated by StrictMath? What to do when inconsistencies are found? And you can specify what's hardcoded. |
public class ExchangeCapsule implements ProtoCapsule<Exchange> {
public long transaction(byte[] sellTokenID, long sellTokenQuant, boolean useStrictMath) {
long supply = 1_000_000_000_000_000_000L;
ExchangeProcessor processor = new ExchangeProcessor(supply, useStrictMath);
ExchangeProcessor strictProcessor = new ExchangeProcessor(supply, true);
long buyTokenQuant = 0;
long strictBuyTokenQuant = 0;
long firstTokenBalance = this.exchange.getFirstTokenBalance();
long secondTokenBalance = this.exchange.getSecondTokenBalance();
if (this.exchange.getFirstTokenId().equals(ByteString.copyFrom(sellTokenID))) {
buyTokenQuant = processor.exchange(firstTokenBalance,
secondTokenBalance,
sellTokenQuant);
strictBuyTokenQuant = strictProcessor.exchange(firstTokenBalance,
secondTokenBalance,
sellTokenQuant);
if (!useStrictMath && buyTokenQuant != strictBuyTokenQuant) {
logAndRecord("{}\t{}\t{}\t{}\t{}", buyTokenQuant, strictBuyTokenQuant, firstTokenBalance, secondTokenBalance, sellTokenQuant); // logAndRecord pow data
}
this.exchange = this.exchange.toBuilder()
.setFirstTokenBalance(firstTokenBalance + sellTokenQuant)
.setSecondTokenBalance(secondTokenBalance - buyTokenQuant)
.build();
} else {
buyTokenQuant = processor.exchange(secondTokenBalance,
firstTokenBalance,
sellTokenQuant);
strictBuyTokenQuant = strictProcessor.exchange(secondTokenBalance,
firstTokenBalance,
sellTokenQuant);
if (!useStrictMath && buyTokenQuant != strictBuyTokenQuant) {
logAndRecord("{}\t{}\t{}\t{}\t{}", buyTokenQuant, strictBuyTokenQuant,secondTokenBalance, firstTokenBalance, sellTokenQuant); // logAndRecord pow data
}
this.exchange = this.exchange.toBuilder()
.setFirstTokenBalance(firstTokenBalance - buyTokenQuant)
.setSecondTokenBalance(secondTokenBalance + sellTokenQuant)
.build();
}
return buyTokenQuant;
}
public class ExchangeProcessor {
private long supply;
private final boolean useStrictMath;
public ExchangeProcessor(long supply, boolean useStrictMath) {
this.supply = supply;
this.useStrictMath = useStrictMath;
}
private long exchangeToSupply(long balance, long quant) {
long newBalance = balance + quant;
double issuedSupply = -supply * (1.0 - Maths.pow(1.0 + (double) quant / newBalance, 0.0005, this.useStrictMath));
long out = (long) issuedSupply;
supply += out;
return out;
}
private long exchangeFromSupply(long balance, long supplyQuant) {
supply -= supplyQuant;
double exchangeBalance = balance * (Maths.pow(1.0 + (double) supplyQuant / supply, 2000.0, this.useStrictMath) - 1.0);
return (long) exchangeBalance;
}
public long exchange(long sellTokenBalance, long buyTokenBalance, long sellTokenQuant) {
long relay = exchangeToSupply(sellTokenBalance, sellTokenQuant);
return exchangeFromSupply(buyTokenBalance, relay);
}
}
private static final Map<Double, Double> powData = Collections.synchronizedMap(new HashMap<>());
public static double pow(double a, double b) {
double strictResult = StrictMath.pow(a, b);
return powData.getOrDefault(a, strictResult);
}
} |
|
If there are other ways to implement X87 Instruction Simulation, please discuss them. |
|
@halibobo1205 I noticed that the pow results in |
Yes, precision loss precisely reduces the amount of the pow data that needs to be hard-coded. |
Background
Java-Tron currently only supports the x86 architecture. Nevertheless, ARM architecture has gained significant traction recently, especially in cloud computing and mobile devices. ARM processors are known for their energy efficiency and cost-effectiveness, making them increasingly popular in data centers, cloud computing, and edge computing scenarios. It will be great to have an option to run Java-Tron using ARM architecture.
Key developments in ARM architecture:
ARM advantages:
Related Issues and PRs
Scope of Impact
The text was updated successfully, but these errors were encountered: