Skip to content

Commit 32344c0

Browse files
Update README.md
1 parent ee7a186 commit 32344c0

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
PHP BPE Text Encoder for GPT-2 / GPT-3
33

44
## About
5-
GPT-2 and GPT-3 use byte pair encoding to turn text into a series of integers to feed into the model. This is a PHP implementation of OpenAI's original python encoder which can be found [here](https://github.com/openai/gpt-2), the main source of inspiration for writing this encoder was the NodeJS version of this encoder, found [here](https://github.com/latitudegames/GPT-3-Encoder).
5+
GPT-2 and GPT-3 use byte pair encoding to turn text into a series of integers to feed into the model. This is a PHP implementation of OpenAI's original python encoder which can be found [here](https://github.com/openai/gpt-2). The main source of inspiration for writing this encoder was the NodeJS version of this encoder, found [here](https://github.com/latitudegames/GPT-3-Encoder).
66

77
You can test the results, by comparing the output generated by this script, with the [official tokenizer page from OpenAI](https://beta.openai.com/tokenizer).
88

@@ -11,7 +11,7 @@ This specific encoder is used in one of my [WordPress plugins](https://coderevol
1111

1212
## Usage
1313

14-
The mbstring PHP extension is needed for this tool to work correctly (in case non-ASCII characters are present in the tokenized text: [details here on how to install mbstring](https://www.php.net/manual/en/mbstring.installation.php)
14+
The mbstring PHP extension is needed for this tool to work correctly (in case non-ASCII characters are present in the tokenized text): [details here on how to install mbstring](https://www.php.net/manual/en/mbstring.installation.php)
1515

1616

1717
```php

0 commit comments

Comments
 (0)