Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(javascript): removing HPS #2001

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -272,11 +272,6 @@ The text of each license is also included in licenses/LICENSE-[project].txt.
java/fury-core/src/main/java/org/apache/fury/type/Generics.java
java/fury-core/src/test/java/org/apache/fury/type/GenericsTest.java

* v8 (https://chromium.googlesource.com/v8/v8.git)
Files:
javascript/packages/hps/src/v8-fast-api-calls.h


================================================================
MIT licenses
================================================================
Expand Down
6 changes: 0 additions & 6 deletions ci/release.py
Original file line number Diff line number Diff line change
Expand Up @@ -148,12 +148,6 @@ def bump_version(**kwargs):
new_version,
_update_js_version,
)
_bump_version(
"javascript/packages/hps",
"package.json",
new_version,
_update_js_version,
)
else:
raise NotImplementedError(f"Unsupported {lang}")

Expand Down
26 changes: 3 additions & 23 deletions docs/guide/xlang_serialization_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,14 +98,7 @@ func main() {
```javascript
import Fury from '@furyjs/fury';

/**
* @furyjs/hps use v8's fast-calls-api that can be called directly by jit, ensure that the version of Node is 20 or above.
* Experimental feature, installation success cannot be guaranteed at this moment
* If you are unable to install the module, replace it with `const hps = null;`
**/
import hps from '@furyjs/hps';

const fury = new Fury({ hps });
const fury = new Fury({});
const input = fury.serialize('hello fury');
const result = fury.deserialize(input);
console.log(result);
Expand Down Expand Up @@ -317,18 +310,11 @@ func main() {
```javascript
import Fury, { Type, InternalSerializerType } from '@furyjs/fury';

/**
* @furyjs/hps use v8's fast-calls-api that can be called directly by jit, ensure that the version of Node is 20 or above.
* Experimental feature, installation success cannot be guaranteed at this moment
* If you are unable to install the module, replace it with `const hps = null;`
**/
import hps from '@furyjs/hps';

// Now we describe data structures using JSON, but in the future, we will use more ways.
const description = Type.object('example.foo', {
foo: Type.string(),
});
const fury = new Fury({ hps });
const fury = new Fury({});
const { serialize, deserialize } = fury.registerSerializer(description);
const input = serialize({ foo: 'hello fury' });
const result = deserialize(input);
Expand Down Expand Up @@ -490,19 +476,13 @@ func main() {

```javascript
import Fury, { Type } from '@furyjs/fury';
/**
* @furyjs/hps use v8's fast-calls-api that can be called directly by jit, ensure that the version of Node is 20 or above.
* Experimental feature, installation success cannot be guaranteed at this moment
* If you are unable to install the module, replace it with `const hps = null;`
**/
import hps from '@furyjs/hps';

const description = Type.object('example.foo', {
foo: Type.string(),
bar: Type.object('example.foo'),
});

const fury = new Fury({ hps });
const fury = new Fury({});
const { serialize, deserialize } = fury.registerSerializer(description);
const data: any = {
foo: 'hello fury',
Expand Down
18 changes: 1 addition & 17 deletions javascript/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,26 +8,18 @@ The Cross-Language part of the protocol is not stable, so the output of this lib

```shell
npm install @furyjs/fury
npm install @furyjs/hps
```

## Usage

```Javascript
import Fury, { Type, InternalSerializerType } from '@furyjs/fury';

/**
* @furyjs/hps use v8's fast-calls-api that can be called directly by jit, ensure that the version of Node is 20 or above.
* Experimental feature, installation success cannot be guaranteed at this moment
* If you are unable to install the module, replace it with `const hps = null;`
**/
import hps from '@furyjs/hps';

// Now we describe data structures using JSON, but in the future, we will use more ways.
const description = Type.object('example.foo', {
foo: Type.string(),
});
const fury = new Fury({ hps });
const fury = new Fury({});
const { serialize, deserialize } = fury.registerSerializer(description);
const input = serialize({ foo: 'hello fury' });
const result = deserialize(input);
Expand All @@ -39,11 +31,3 @@ console.log(result);
### fury

Fury protocol implementation. It generates JavaScript code at runtime to make sure that all the code could be optimized by v8 JIT efficiently.

### hps

Node.js high-performance suite, ensuring that your Node.js version is 20 or later.

`hps` is use for detect the string type in v8. Fury support latin1 and utf8 string both, we should get the certain type of string before write it
in buffer. It is slow to detect the string is latin1 or utf8, but hps can detect it by a hack way, which is called FASTCALL in v8.
so it is not stable now.
3 changes: 1 addition & 2 deletions javascript/benchmark/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@

const Fury = require("@furyjs/fury");
const utils = require("@furyjs/fury/dist/lib/util");
const hps = require('@furyjs/hps');
const fury = new Fury.default({ hps, refTracking: false, useSliceString: true });
const fury = new Fury.default({ refTracking: false, useSliceString: true });
const Benchmark = require("benchmark");
const protobuf = require("protobufjs");
const path = require('path');
Expand Down
3 changes: 1 addition & 2 deletions javascript/benchmark/map.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,8 @@

const Fury = require("@furyjs/fury");
const beautify = require("js-beautify");
const hps = require('@furyjs/hps');
const fury = new Fury.default({
hps, refTracking: false, useSliceString: true, hooks: {
refTracking: false, useSliceString: true, hooks: {
afterCodeGenerated: (code) => {
return beautify.js(code, { indent_size: 2, space_in_empty_paren: true, indent_empty_lines: true });
}
Expand Down
6 changes: 1 addition & 5 deletions javascript/jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,9 @@

/** @type {import('ts-jest').JestConfigWithTsJest} */
const semver = require("semver");
const hpsEnable = semver.gt(process.versions.node, '20.0.0') && process.platform !== 'win32';

module.exports = {
collectCoverage: hpsEnable,
collectCoverage: true,
preset: 'ts-jest',
testEnvironment: 'node',
collectCoverageFrom: [
Expand All @@ -31,9 +30,6 @@ module.exports = {
"!**/build/**",
"!packages/fury/lib/murmurHash3.ts"
],
"testPathIgnorePatterns" : [
hpsEnable ? null : "(.*)/hps.test.ts$",
].filter(Boolean),
transform: {
'\\.ts$': ['ts-jest', {
tsconfig: {
Expand Down
5 changes: 2 additions & 3 deletions javascript/package.json
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
{
"scripts": {
"test": "npm run build && jest",
"clear": "rm -rf ./packages/fury/dist && rm -rf ./packages/hps/dist",
"build": "npm run clear && npm run build -w packages/fury -w packages/hps",
"clear": "rm -rf ./packages/fury/dist",
"build": "npm run clear && npm run build -w packages/fury",
"lint": "eslint .",
"lint-fix": "eslint . --fix"
},
"repository": "[email protected]:apache/fury.git",
"workspaces": [
"packages/hps",
"packages/fury"
],
"devDependencies": {
Expand Down
6 changes: 0 additions & 6 deletions javascript/packages/fury/lib/type.ts
Original file line number Diff line number Diff line change
Expand Up @@ -95,18 +95,12 @@ export const HalfMinInt32 = MinInt32 / 2;
export const LATIN1 = 0;
export const UTF8 = 1;

export interface Hps {
isLatin1: (str: string) => boolean;
stringCopy: (str: string, dist: Uint8Array, offset: number) => void;
}

export enum Mode {
SchemaConsistent,
Compatible,
}

export interface Config {
hps?: Hps;
refTracking?: boolean;
useSliceString?: boolean;
hooks?: {
Expand Down
14 changes: 8 additions & 6 deletions javascript/packages/fury/lib/writer/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ import { PlatformBuffer, alloc, strByteLength } from "../platformBuffer";
import { OwnershipError } from "../error";
import { toFloat16 } from "./number";

declare global {
// eslint-disable-next-line
var isOneByteString: ((input: string) => boolean) | undefined;
}

const MAX_POOL_SIZE = 1024 * 1024 * 3; // 3MB

export class BinaryWriter {
Expand All @@ -32,12 +37,10 @@ export class BinaryWriter {
private reserved = 0;
private locked = false;
private config: Config;
private hpsEnable = false;

constructor(config: Config) {
this.initPoll();
this.config = config;
this.hpsEnable = Boolean(config?.hps);
}

private initPoll() {
Expand Down Expand Up @@ -188,14 +191,13 @@ export class BinaryWriter {
}

stringOfVarUInt32Fast(v: string) {
const { isLatin1: detectIsLatin1, stringCopy } = this.config.hps!;
const isLatin1 = detectIsLatin1(v);
const isLatin1 = globalThis.isOneByteString!(v);
const len = isLatin1 ? v.length : strByteLength(v);
this.dataView.setUint8(this.cursor++, isLatin1 ? LATIN1 : UTF8);
this.varUInt32(len);
this.reserve(len);
if (isLatin1) {
stringCopy(v, this.platformBuffer, this.cursor);
this.platformBuffer.write(v, this.cursor, "latin1");
} else {
if (len < 40) {
this.fastWriteStringUtf8(v, this.platformBuffer, this.cursor);
Expand Down Expand Up @@ -335,7 +337,7 @@ export class BinaryWriter {
}

stringOfVarUInt32(v: string) {
return this.hpsEnable
return globalThis.isOneByteString
? this.stringOfVarUInt32Fast(v)
: this.stringOfVarUInt32Slow(v);
}
Expand Down
1 change: 0 additions & 1 deletion javascript/packages/fury/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,5 @@
},
"repository": "[email protected]:apache/fury.git",
"workspaces": [
"packages/hps"
]
}
18 changes: 0 additions & 18 deletions javascript/packages/hps/README.md

This file was deleted.

32 changes: 0 additions & 32 deletions javascript/packages/hps/binding.gyp

This file was deleted.

32 changes: 0 additions & 32 deletions javascript/packages/hps/index.ts

This file was deleted.

26 changes: 0 additions & 26 deletions javascript/packages/hps/package.json

This file was deleted.

Loading
Loading