diff --git a/Documentation/Classes/OpenAI.md b/Documentation/Classes/OpenAI.md index 1fd31ec..9b07525 100644 --- a/Documentation/Classes/OpenAI.md +++ b/Documentation/Classes/OpenAI.md @@ -63,6 +63,7 @@ The API provides access to multiple resources that allow seamless interaction wi | `moderations` | [OpenAIModerationsAPI](OpenAIModerationsAPI.md) | Access to the Moderations API. | | `embeddings` | [OpenAIEmbeddingsAPI](OpenAIEmbeddingsAPI.md) | Access to the Embeddings API. | | `files` | [OpenAIFilesAPI](OpenAIFilesAPI.md) | Access to the Files API. | +| `uploads` | [OpenAIUploadsAPI](OpenAIUploadsAPI.md) | Access to the Uploads API. | ### Example Usage diff --git a/Documentation/Classes/OpenAIUpload.md b/Documentation/Classes/OpenAIUpload.md new file mode 100644 index 0000000..62db789 --- /dev/null +++ b/Documentation/Classes/OpenAIUpload.md @@ -0,0 +1,59 @@ +# OpenAIUpload + +## Description +Represents a multipart file upload object in OpenAI. The Upload object allows you to upload large files (up to 8 GB) by breaking them into multiple parts. + +## Properties + +| Property | Type | Description | +| -------- | ---- | ----------- | +| `id` | Text | The Upload unique identifier, which can be referenced in API endpoints. | +| `object` | Text | The object type, which is always "upload". | +| `bytes` | Integer | The intended number of bytes to be uploaded. | +| `created_at` | Integer | The Unix timestamp (in seconds) for when the Upload was created. | +| `filename` | Text | The name of the file to be uploaded. | +| `purpose` | Text | The intended purpose of the file. Possible values: `assistants`, `batch`, `fine-tune`, `vision`, `user_data`. | +| `status` | Text | The status of the Upload. Possible values: `pending`, `completed`, `cancelled`, `expired`. | +| `expires_at` | Integer | The Unix timestamp (in seconds) for when the Upload will expire. | +| `file` | cs.AIKit.OpenAIFile | The ready File object after the Upload is completed. Only present when status is "completed". | +| `mime_type` | Text | The MIME type of the file (e.g., "text/jsonl", "image/png", "application/pdf"). Could be returned empty by API.| + +## Constructor + +```4d +$upload:=cs.AIKit.OpenAIUpload.new($object) +``` + +**Parameters:** +- `$object` (Object): Object containing upload properties + +**Note:** This class is typically instantiated by the API response, not manually by users. + +## Example + +```4d +// After creating and completing an upload +var $result : cs.AIKit.OpenAIUploadResult +$result:=$client.uploads.complete($uploadId; $params) + +If ($result.success) + var $upload : cs.AIKit.OpenAIUpload + $upload:=$result.upload + + ALERT("Upload ID: "+$upload.id) + ALERT("Status: "+$upload.status) + ALERT("Filename: "+$upload.filename) + + If ($upload.status="completed") && ($upload.file#Null) + ALERT("File ID: "+$upload.file.id) + ALERT("File ready for use!") + End if +End if +``` + +## See Also +- [OpenAIUploadResult](OpenAIUploadResult.md) +- [OpenAIUploadsAPI](OpenAIUploadsAPI.md) +- [OpenAIUploadParameters](OpenAIUploadParameters.md) +- [OpenAIFile](OpenAIFile.md) +- [OpenAI Uploads API Documentation](https://platform.openai.com/docs/api-reference/uploads) diff --git a/Documentation/Classes/OpenAIUploadCompleteParameters.md b/Documentation/Classes/OpenAIUploadCompleteParameters.md new file mode 100644 index 0000000..417d162 --- /dev/null +++ b/Documentation/Classes/OpenAIUploadCompleteParameters.md @@ -0,0 +1,35 @@ +# OpenAIUploadCompleteParameters + +## Description +Optional parameters for completing an Upload. + +## Inherits + +[OpenAIParameters](OpenAIParameters.md) + +## Properties + +| Property | Type | Required | Default | Description | +| -------- | ---- | -------- | ------- | ----------- | +| `md5` | Text | No | `Null` | The optional MD5 checksum for the file contents to verify if the bytes uploaded matches what you expect. This provides an additional verification step to ensure file integrity. | + +## Example + +```4d +VAR $uploadId:="upload_abc123" +VAR $partIds:=["part_def456"; "part_ghi789"; "part_jkl012"] + +$md5Hash:="5d41402abc4b2a76b9719d911017c592" // Example MD5 +var $completeParams:=cs.AIKit.OpenAIUploadCompleteParameters.new() +$completeParams.md5:=$md5Hash // Optional verification + +// Complete the upload - part_ids is passed as explicit parameter +$result:=$client.uploads.complete($uploadId; $partIds; $completeParams) +``` + +## See Also +- [OpenAIUpload](OpenAIUpload.md) +- [OpenAIUploadResult](OpenAIUploadResult.md) +- [OpenAIUploadPart](OpenAIUploadPart.md) +- [OpenAIUploadsAPI](OpenAIUploadsAPI.md) +- [OpenAI Uploads API Documentation](https://platform.openai.com/docs/api-reference/uploads/complete) diff --git a/Documentation/Classes/OpenAIUploadParameters.md b/Documentation/Classes/OpenAIUploadParameters.md new file mode 100644 index 0000000..e23f553 --- /dev/null +++ b/Documentation/Classes/OpenAIUploadParameters.md @@ -0,0 +1,36 @@ +# OpenAIUploadParameters + +## Description +Optional parameters for creating an Upload object in OpenAI. + +## Inherits + +[OpenAIParameters](OpenAIParameters.md) + +## Properties + +| Property | Type | Required | Default | Description | +| -------- | ---- | -------- | ------- | ----------- | +| `expires_after` | Object | No | `Null` | The expiration policy for a file. By default, files with `purpose=batch` expire after 30 days and all other files are persisted until they are manually deleted. Object structure: `{anchor: "created_at", seconds: 3600}` where `seconds` must be between 3600 (1 hour) and 2592000 (30 days). | + + +## Example + +```4d +var $params:=cs.AIKit.OpenAIUploadParameters.new({\ + expires_after: {\ + anchor: "created_at"; \ + seconds: 7200\ + }\ +}) + +// Mandatory parameters are passed to the function +$result:=$client.uploads.create("large_dataset.jsonl"; 2147483648; "batch"; "text/jsonl"; $params) +``` + +## See Also +- [OpenAIUpload](OpenAIUpload.md) +- [OpenAIUploadResult](OpenAIUploadResult.md) +- [OpenAIUploadsAPI](OpenAIUploadsAPI.md) +- [OpenAIParameters](OpenAIParameters.md) +- [OpenAI Uploads API Documentation](https://platform.openai.com/docs/api-reference/uploads/create) diff --git a/Documentation/Classes/OpenAIUploadPart.md b/Documentation/Classes/OpenAIUploadPart.md new file mode 100644 index 0000000..3b466ff --- /dev/null +++ b/Documentation/Classes/OpenAIUploadPart.md @@ -0,0 +1,19 @@ +# OpenAIUploadPart + +## Description +Represents a chunk of bytes added to an Upload object. Each Part can be up to 64 MB in size, and multiple Parts can be uploaded in parallel. + +## Properties + +| Property | Type | Description | +| -------- | ---- | ----------- | +| `id` | Text | The upload Part unique identifier, which can be referenced in API endpoints. | +| `object` | Text | The object type, which is always "upload.part". | +| `created_at` | Integer | The Unix timestamp (in seconds) for when the Part was created. | +| `upload_id` | Text | The ID of the Upload object that this Part was added to. | + +## See Also +- [OpenAIUploadPartResult](OpenAIUploadPartResult.md) +- [OpenAIUploadsAPI](OpenAIUploadsAPI.md) +- [OpenAIUpload](OpenAIUpload.md) +- [OpenAI Uploads API Documentation](https://platform.openai.com/docs/api-reference/uploads/add-part) diff --git a/Documentation/Classes/OpenAIUploadPartResult.md b/Documentation/Classes/OpenAIUploadPartResult.md new file mode 100644 index 0000000..ff03a14 --- /dev/null +++ b/Documentation/Classes/OpenAIUploadPartResult.md @@ -0,0 +1,19 @@ +# OpenAIUploadPartResult + +## Description +Result class for Upload Part operations. Contains the upload part object returned by the API after adding a part to an upload, along with success/error status. + +## Inherits + +[OpenAIResult](OpenAIResult.md) + +## Properties + +| Property | Type | Description | +| -------- | ---- | ----------- | +| `part` | cs.AIKit.OpenAIUploadPart | (read-only) Returns the upload part object from the API response. Returns Null if the response is invalid. | + +## See Also +- [OpenAIUploadPart](OpenAIUploadPart.md) +- [OpenAIUploadsAPI](OpenAIUploadsAPI.md) +- [OpenAI Uploads API Documentation](https://platform.openai.com/docs/api-reference/uploads/add-part) diff --git a/Documentation/Classes/OpenAIUploadResult.md b/Documentation/Classes/OpenAIUploadResult.md new file mode 100644 index 0000000..41917c8 --- /dev/null +++ b/Documentation/Classes/OpenAIUploadResult.md @@ -0,0 +1,19 @@ +# OpenAIUploadResult + +## Description +Result class for Upload operations (create, complete, or cancel). Contains the upload object returned by the API along with success/error status. + +## Inherits + +[OpenAIResult](OpenAIResult.md) + +## Properties + +| Property | Type | Description | +| -------- | ---- | ----------- | +| `upload` | cs.AIKit.OpenAIUpload | Returns the upload object from the API response. Returns Null if the response is invalid. | + +## See Also +- [OpenAIUpload](OpenAIUpload.md) +- [OpenAIUploadsAPI](OpenAIUploadsAPI.md) +- [OpenAI Uploads API Documentation](https://platform.openai.com/docs/api-reference/uploads) diff --git a/Documentation/Classes/OpenAIUploadsAPI.md b/Documentation/Classes/OpenAIUploadsAPI.md new file mode 100644 index 0000000..8d0787a --- /dev/null +++ b/Documentation/Classes/OpenAIUploadsAPI.md @@ -0,0 +1,278 @@ +# OpenAIUploadsAPI + +API Reference: + +## Description +API resource for managing large file uploads in OpenAI. The Uploads API allows you to upload files in multiple parts, supporting files up to 8 GB in size. This is particularly useful for handling large training datasets, batch files, or other large file uploads. + +## Inherits + +[OpenAIAPIResource](OpenAIAPIResource.md) + +## Overview +The Uploads API workflow consists of three main steps: +1. **Create** an Upload object with metadata about the file +2. **Add Parts** to the Upload (chunks of up to 64 MB each) +3. **Complete** or **Cancel** the Upload + +Once completed, the Upload creates a usable File object that can be used across the OpenAI platform. + +## Key Features +- Upload files up to 8 GB in total size +- Each part can be up to 64 MB +- Supports parallel part uploads +- Custom ordering of parts when completing +- Automatic expiration after 1 hour if not completed +- Optional MD5 checksum verification + + +## Methods + +### create + +Creates an intermediate Upload object that you can add Parts to. + +**create**(*filename* : Text ; *bytes* : Integer ; *purpose* : Text ; *mimeType* : Text ; *parameters* : [OpenAIUploadParameters](OpenAIUploadParameters.md)) : [OpenAIUploadResult](OpenAIUploadResult.md) + +| Parameter | Type | Description | +|-----------------|-------------------------------------------|-------------------------------------------| +| *filename* | Text | **Required.** The name of the file to upload | +| *bytes* | Integer | **Required.** The number of bytes in the file you are uploading | +| *purpose* | Text | **Required.** The intended purpose of the uploaded file | +| *mimeType* | Text | **Required.** The MIME type of the file | +| *parameters* | [OpenAIUploadParameters](OpenAIUploadParameters.md) | Optional parameters including expires_after | +| Function result | [OpenAIUploadResult](OpenAIUploadResult.md) | Result containing the Upload object with status "pending" | + +**Throws:** An error if `filename` is empty, `bytes` is not positive, `purpose` is empty, or `mimeType` is empty. + +**Example:** +```4d +var $client : cs.AIKit.OpenAI +var $params : cs.AIKit.OpenAIUploadParameters +var $result : cs.AIKit.OpenAIUploadResult + +$client:=cs.AIKit.OpenAI.new() + +// Optional: Set expiration policy +$params:=cs.AIKit.OpenAIUploadParameters.new() +$params.expires_after:={} +$params.expires_after.anchor:="created_at" +$params.expires_after.seconds:=3600 // Expire after 1 hour + +// Create the upload +$result:=$client.uploads.create("training_data.jsonl"; 2147483648; "fine-tune"; "text/jsonl"; $params) + +If ($result.success) + $upload:=$result.upload + ALERT("Upload created: "+$upload.id) +Else + ALERT("Error: "+$result.error.message) +End if +``` + +--- + +### addPart + +Adds a Part (chunk of bytes) to an Upload object. + +**addPart**(*uploadId* : Text ; *data* : [4D.File](https://developer.4d.com/docs/API/FileClass) or [4D.Blob](https://developer.4d.com/docs/API/BlobClass) ; *parameters* : [OpenAIParameters](OpenAIParameters.md)) : [OpenAIUploadPartResult](OpenAIUploadPartResult.md) + +| Parameter | Type | Description | +|-----------------|-------------------------------------------|-------------------------------------------| +| *uploadId* | Text | **Required.** The ID of the Upload | +| *data* | [4D.File](https://developer.4d.com/docs/API/FileClass) or [4D.Blob](https://developer.4d.com/docs/API/BlobClass) | **Required.** The chunk of bytes for this Part | +| *parameters* | [OpenAIParameters](OpenAIParameters.md) | Optional parameters | +| Function result | [OpenAIUploadPartResult](OpenAIUploadPartResult.md) | Result containing the upload Part object | + +**Throws:** An error if `uploadId` is empty or if `data` is not a 4D.File or 4D.Blob. + +**Notes:** +- Each Part can be at most 64 MB +- Parts can be added in parallel +- The order is specified when completing the upload + +**Example:** +```4d +var $uploadId:="upload_abc123" +var $partIds:=[] + +// Add first part +var $partFile:=Folder(fk desktop folder).file("part1.bin") +$result:=$client.uploads.addPart($uploadId; $partFile) + +If ($result.success) + $part:=$result.part + $partIds.push($part.id) +End if + +// Add second part +$partFile:=Folder(fk desktop folder).file("part2.bin") +var $result:=$client.uploads.addPart($uploadId; $partFile) + +If ($result.success) + $part:=$result.part + $partIds.push($part.id) +End if +``` + +--- + +### complete + +Completes the Upload and creates a usable File object. + +**complete**(*uploadId* : Text ; *part_ids* : Collection ; *parameters* : [OpenAIUploadCompleteParameters](OpenAIUploadCompleteParameters.md)) : [OpenAIUploadResult](OpenAIUploadResult.md) + +| Parameter | Type | Description | +|-----------------|-------------------------------------------|-------------------------------------------| +| *uploadId* | Text | **Required.** The ID of the Upload | +| *part_ids* | Collection | **Required.** The ordered list of Part IDs | +| *parameters* | [OpenAIUploadCompleteParameters](OpenAIUploadCompleteParameters.md) | Optional parameters including md5 checksum | +| Function result | [OpenAIUploadResult](OpenAIUploadResult.md) | Result with status "completed" and a file property | + +**Throws:** An error if `uploadId` is empty or if `part_ids` is null or empty. + +**Notes:** +- Must specify the ordered list of Part IDs +- Total bytes must match the initially specified amount +- No Parts may be added after completion + +**Example:** +```4d +var $result : cs.AIKit.OpenAIUploadResult +var $params : cs.AIKit.OpenAIUploadCompleteParameters + +// Optional: Add MD5 checksum for verification +$params:=cs.AIKit.OpenAIUploadCompleteParameters.new() +$params.md5:="d41d8cd98f00b204e9800998ecf8427e" + +$result:=$client.uploads.complete($uploadId; $partIds; $params) + +If ($result.success) + $upload:=$result.upload + If ($upload.status="completed") + $file:=$upload.file + ALERT("File ready: "+$file.id) + End if +End if +``` + +--- + +### cancel + +Cancels the Upload. No Parts may be added after cancellation. + +**cancel**(*uploadId* : Text ; *parameters* : [OpenAIParameters](OpenAIParameters.md)) : [OpenAIUploadResult](OpenAIUploadResult.md) + +| Parameter | Type | Description | +|-----------------|-------------------------------------------|-------------------------------------------| +| *uploadId* | Text | **Required.** The ID of the Upload | +| *parameters* | [OpenAIParameters](OpenAIParameters.md) | Optional parameters | +| Function result | [OpenAIUploadResult](OpenAIUploadResult.md) | Result containing the Upload object with status "cancelled" | + +**Throws:** An error if `uploadId` is empty. + +**Example:** +```4d +var $result:=$client.uploads.cancel($uploadId) + +If ($result.success) + $upload:=$result.upload + ASSERT($upload.status="cancelled") +End if +``` + +--- + +## Complete Upload Workflow Example + +```4d +// 1. Create the upload + +var $params:=cs.AIKit.OpenAIUploadParameters.new() +$params.expires_after:={} +$params.expires_after.anchor:="created_at" +$params.expires_after.seconds:=3600 // Expire after 1 hour + +var $result: cs.AIKit.OpenAIUploadResult:=$client.uploads.create("large_dataset.jsonl"; 134217728; "fine-tune"; "text/jsonl"; $params) +If (Not($result.success)) + // Handle error + return +End if + +var $uploadId:=$result.upload.id +var $partIds:=[] + +// 2. Split file and upload parts +var $sourceFile : 4D.File:=Folder(fk desktop folder).file("large_dataset.jsonl") +var $chunkSize : Integer:=67108864 // 64 MB chunks +var $offset : Integer:=0 +var $partNumber : Integer:=1 + +While ($offset<$sourceFile.size) + // Read chunk from file + var $blob : 4D.Blob:=4D.Blob.new() + var $bytesToRead : Integer:=Min($chunkSize; $sourceFile.size-$offset) + + // In real implementation, you would read a chunk of the file here + // For example using File.getContent() or BLOB operations + + // Upload the part + var $partResult : cs.AIKit.OpenAIUploadPartResult + $partResult:=$client.uploads.addPart($uploadId; $blob) + + If ($partResult.success) + $partIds.push($partResult.part.id) + $offset:=$offset+$bytesToRead + $partNumber:=$partNumber+1 + Else + // Handle error - maybe cancel the upload + $client.uploads.cancel($uploadId) + return + End if +End while + +// 3. Complete the upload +var $completeParams:=cs.AIKit.OpenAIUploadCompleteParameters.new() +$result:=$client.uploads.complete($uploadId; $partIds; $completeParams) + +If ($result.success && ($result.upload.status="completed")) + var $file : cs.AIKit.OpenAIFile:=$result.upload.file + ALERT("Upload completed! File ID: "+$file.id) + + // Now you can use this file for fine-tuning or other purposes +Else + ALERT("Upload failed to complete") +End if +``` + +## Supported MIME Types by Purpose + +| Purpose | Supported MIME Types | +|---------|---------------------| +| assistants | text/*, application/json, application/pdf, image/*, etc. | +| batch | application/jsonl (max 200 MB) | +| fine-tune | application/jsonl, text/jsonl | +| vision | image/jpeg, image/png, image/gif, image/webp | + +## Important Notes + +1. **Upload Expiration**: Uploads expire after 1 hour of creation if not completed +2. **Size Limits**: + - Maximum upload size: 8 GB + - Maximum part size: 64 MB + - Batch API: 200 MB maximum for .jsonl files +3. **Parallel Uploads**: Parts can be uploaded in parallel for faster processing +4. **Byte Count**: The total bytes uploaded must exactly match the `bytes` specified when creating the upload +5. **Part Ordering**: Specify the correct order of parts when completing the upload + +## See Also +- [OpenAIUpload](OpenAIUpload.md) - The Upload object model +- [OpenAIUploadPart](OpenAIUploadPart.md) - The Upload Part object model +- [OpenAIUploadParameters](OpenAIUploadParameters.md) - Create upload parameters +- [OpenAIUploadCompleteParameters](OpenAIUploadCompleteParameters.md) - Complete upload parameters +- [OpenAIUploadResult](OpenAIUploadResult.md) - Upload result class +- [OpenAIFilesAPI](OpenAIFilesAPI.md) - Regular file upload API +- [OpenAI Platform Documentation](https://platform.openai.com/docs/api-reference/uploads) diff --git a/Project/Sources/Classes/OpenAI.4dm b/Project/Sources/Classes/OpenAI.4dm index 77c8a9f..9c0af85 100644 --- a/Project/Sources/Classes/OpenAI.4dm +++ b/Project/Sources/Classes/OpenAI.4dm @@ -14,7 +14,7 @@ property models : cs:C1710.OpenAIModelsAPI // property fineTunings : cs.OpenAIFineTuningsAPI // property beta : cs.OpenAIBetaAPI // property batches : cs.OpenAIBatchesAPI -// property uploads : cs.OpenAIUploadsAPI +property uploads : cs:C1710.OpenAIUploadsAPI // MARK: account options property apiKey : Text:="" @@ -98,6 +98,7 @@ Class constructor( ... : Variant) // This.audio:=cs.OpenAIAudioAPI.new(This) This:C1470.moderations:=cs:C1710.OpenAIModerationsAPI.new(This:C1470) This:C1470.models:=cs:C1710.OpenAIModelsAPI.new(This:C1470) + This:C1470.uploads:=cs:C1710.OpenAIUploadsAPI.new(This:C1470) If (Count parameters:C259=0) This:C1470._fillDefaultParameters() @@ -481,4 +482,4 @@ Function _formData($body : Object; $files : Object; $boundary : Text) : 4D:C1709 TEXT TO BLOB:C554($textPart; $temp; UTF8 text without length:K22:17) COPY BLOB:C558($temp; $result; 0; BLOB size:C605($result); BLOB size:C605($temp)) - return $result \ No newline at end of file + return $result diff --git a/Project/Sources/Classes/OpenAIUpload.4dm b/Project/Sources/Classes/OpenAIUpload.4dm new file mode 100644 index 0000000..1ea730b --- /dev/null +++ b/Project/Sources/Classes/OpenAIUpload.4dm @@ -0,0 +1,47 @@ +// The Upload object represents a multipart file upload + +// The Upload unique identifier, which can be referenced in API endpoints +property id : Text + +// The object type, which is always "upload" +property object : Text + +// The intended number of bytes to be uploaded +property bytes : Integer + +// The Unix timestamp (in seconds) for when the Upload was created +property created_at : Integer + +// The name of the file to be uploaded +property filename : Text + +// The intended purpose of the file (assistants, batch, fine-tune, vision, user_data, etc.) +property purpose : Text + +// The status of the Upload (pending, completed, cancelled, or expired) +property status : Text + +// The Unix timestamp (in seconds) for when the Upload will expire +property expires_at : Integer + +// The ready File object after the Upload is completed +property file : cs:C1710.OpenAIFile + +// The MIME type of the file +property mime_type : Text + +Class constructor($object : Object) + If ($object=Null:C1517) + return + End if + + var $key : Text + For each ($key; $object) + Case of + : ($key="file") && ($object[$key]#Null:C1517) + // Create nested File object + This:C1470.file:=cs:C1710.OpenAIFile.new($object[$key]) + Else + This:C1470[$key]:=$object[$key] + End case + End for each diff --git a/Project/Sources/Classes/OpenAIUploadCompleteParameters.4dm b/Project/Sources/Classes/OpenAIUploadCompleteParameters.4dm new file mode 100644 index 0000000..9904086 --- /dev/null +++ b/Project/Sources/Classes/OpenAIUploadCompleteParameters.4dm @@ -0,0 +1,28 @@ +// Parameters for completing an Upload +// Note: Mandatory parameter (part_ids) is passed as an explicit function parameter + +// Internal property (set by API function, do not set manually) +property part_ids : Collection + +// Optional: The md5 checksum for the file contents to verify if the bytes uploaded matches what you expect +property md5 : Text + +Class extends OpenAIParameters + +Class constructor($object : Object) + Super:C1705($object) + +Function body() : Object + var $body : Object:=Super:C1706.body() + + // Required parameter (set by API function) + If (This:C1470.part_ids#Null:C1517) + $body.part_ids:=This:C1470.part_ids + End if + + // Optional: MD5 checksum + If (Length:C16(This:C1470.md5)>0) + $body.md5:=This:C1470.md5 + End if + + return $body diff --git a/Project/Sources/Classes/OpenAIUploadParameters.4dm b/Project/Sources/Classes/OpenAIUploadParameters.4dm new file mode 100644 index 0000000..638aa0c --- /dev/null +++ b/Project/Sources/Classes/OpenAIUploadParameters.4dm @@ -0,0 +1,30 @@ + +// The expiration policy for a file. By default, files with purpose=batch expire after 30 days +// and all other files are persisted until they are manually deleted. +property expires_after : Object + +Class extends OpenAIParameters + +Class constructor($object : Object) + Super:C1705($object) + +Function body() : Object + var $body : Object:=Super:C1706.body() + + // Optional expiration policy + If (This:C1470.expires_after#Null:C1517) + $body.expires_after:={} + + // Anchor timestamp after which the expiration policy applies. Supported anchors: created_at. + If (Length:C16(String:C10(This:C1470.expires_after.anchor))>0) + $body.expires_after.anchor:=This:C1470.expires_after.anchor + End if + + // The number of seconds after the anchor time that the file will expire. Must be between 3600 (1 hour) and 2592000 (30 days). + If (This:C1470.expires_after.seconds>0) + $body.expires_after.seconds:=This:C1470.expires_after.seconds + End if + End if + + return $body + \ No newline at end of file diff --git a/Project/Sources/Classes/OpenAIUploadPart.4dm b/Project/Sources/Classes/OpenAIUploadPart.4dm new file mode 100644 index 0000000..bf94116 --- /dev/null +++ b/Project/Sources/Classes/OpenAIUploadPart.4dm @@ -0,0 +1,23 @@ +// The upload Part represents a chunk of bytes added to an Upload object + +// The upload Part unique identifier, which can be referenced in API endpoints +property id : Text + +// The object type, which is always "upload.part" +property object : Text + +// The Unix timestamp (in seconds) for when the Part was created +property created_at : Integer + +// The ID of the Upload object that this Part was added to +property upload_id : Text + +Class constructor($object : Object) + If ($object=Null:C1517) + return + End if + + var $key : Text + For each ($key; $object) + This:C1470[$key]:=$object[$key] + End for each diff --git a/Project/Sources/Classes/OpenAIUploadPartResult.4dm b/Project/Sources/Classes/OpenAIUploadPartResult.4dm new file mode 100644 index 0000000..6dcbc0b --- /dev/null +++ b/Project/Sources/Classes/OpenAIUploadPartResult.4dm @@ -0,0 +1,22 @@ +// Result class for Upload Part operations (add part) +Class extends OpenAIResult + +/* +* Returns the upload part object from the API response +* @return {cs.OpenAIUploadPart} The upload part object, or Null if invalid response +*/ +Function get part : cs:C1710.OpenAIUploadPart + var $body:=This:C1470._objectBody() + If (($body=Null:C1517) || (Not:C34(Value type:C1509($body.id)=Is text:K8:3))) + return Null:C1517 + End if + + var $part:=Try(cs:C1710.OpenAIUploadPart.new($body)) + If ($part=Null:C1517) + var $errors:=Last errors:C1799 + If (($errors#Null:C1517) && (This:C1470.errors=Null:C1517)) + This:C1470._errors:=$errors // decoding error + End if + End if + + return $part diff --git a/Project/Sources/Classes/OpenAIUploadResult.4dm b/Project/Sources/Classes/OpenAIUploadResult.4dm new file mode 100644 index 0000000..47dd7c6 --- /dev/null +++ b/Project/Sources/Classes/OpenAIUploadResult.4dm @@ -0,0 +1,22 @@ +// Result class for Upload operations (create, complete, or cancel) +Class extends OpenAIResult + +/* +* Returns the upload object from the API response +* @return {cs.OpenAIUpload} The upload object, or Null if invalid response +*/ +Function get upload : cs:C1710.OpenAIUpload + var $body:=This:C1470._objectBody() + If (($body=Null:C1517) || (Not:C34(Value type:C1509($body.id)=Is text:K8:3))) + return Null:C1517 + End if + + var $upload:=Try(cs:C1710.OpenAIUpload.new($body)) + If ($upload=Null:C1517) + var $errors:=Last errors:C1799 + If (($errors#Null:C1517) && (This:C1470.errors=Null:C1517)) + This:C1470._errors:=$errors // decoding error + End if + End if + + return $upload diff --git a/Project/Sources/Classes/OpenAIUploadsAPI.4dm b/Project/Sources/Classes/OpenAIUploadsAPI.4dm new file mode 100644 index 0000000..5451c00 --- /dev/null +++ b/Project/Sources/Classes/OpenAIUploadsAPI.4dm @@ -0,0 +1,153 @@ +// API resource for managing file uploads in OpenAI +// Allows you to upload large files in multiple parts (up to 8 GB) +Class extends OpenAIAPIResource + +Class constructor($client : cs:C1710.OpenAI) + Super:C1705($client) + +/* +* Creates an intermediate Upload object that you can add Parts to. +* Currently, an Upload can accept at most 8 GB in total and expires after an hour after you create it. +* +* Once you complete the Upload, a File object will be created that contains all the parts you uploaded. +* This File is usable in the rest of the platform as a regular File object. +* +* @param $filename {Text} The name of the file to upload (required) +* @param $bytes {Integer} The number of bytes in the file you are uploading (required) +* @param $purpose {Text} The intended purpose of the uploaded file (required) +* @param $mime_Type {Text} The MIME type of the file (required) +* @param $parameters {cs.OpenAIUploadParameters} Optional parameters including expires_after +* @return {cs.OpenAIUploadResult} Result containing the Upload object with status pending +* @throws Error if any required parameter is empty or invalid +*/ +Function create($filename : Text; $bytes : Integer; $purpose : Text; $mimeType : Text; $parameters : cs:C1710.OpenAIUploadParameters) : cs:C1710.OpenAIUploadResult + + // Validate required parameters + If (Length:C16($filename)=0) + throw:C1805(1; "Expected a non-empty value for `filename`") + End if + + If ($bytes<=0) + throw:C1805(1; "Expected a positive value for `bytes`") + End if + + If (Length:C16($purpose)=0) + throw:C1805(1; "Expected a non-empty value for `purpose`") + End if + + If (Length:C16($mimeType)=0) + throw:C1805(1; "Expected a non-empty value for `mime_type`") + End if + + If (Not:C34(OB Instance of:C1731($parameters; cs:C1710.OpenAIUploadParameters))) + $parameters:=cs:C1710.OpenAIUploadParameters.new($parameters) + End if + + // Set required parameters + var $body : Object:=$parameters.body() + $body.filename:=$filename + $body.bytes:=$bytes + $body.purpose:=$purpose + $body.mime_type:=$mimeType + + return This:C1470._client._post("/uploads"; $body; $parameters; cs:C1710.OpenAIUploadResult) + +/* +* Adds a Part to an Upload object. A Part represents a chunk of bytes from the file you are trying to upload. +* +* Each Part can be at most 64 MB, and you can add Parts until you hit the Upload maximum of 8 GB. +* It is possible to add multiple Parts in parallel. You can decide the intended order of the Parts when you complete the Upload. +* +* @param $uploadId {Text} The ID of the Upload (required) +* @param $data {4D.File|4D.Blob} The chunk of bytes for this Part (required) +* @param $parameters {cs.OpenAIParameters} Optional parameters for the request +* @return {cs.OpenAIUploadPartResult} Result containing the upload Part object +* @throws Error if uploadId is empty or data is invalid +*/ +Function addPart($uploadId : Text; $data : Variant; $parameters : cs:C1710.OpenAIParameters) : cs:C1710.OpenAIUploadPartResult + If (Length:C16($uploadId)=0) + throw:C1805(1; "Expected a non-empty value for `uploadId`") + End if + + // Validate data parameter - must be either 4D.File or 4D.Blob + var $isFile:=False:C215 + var $isBlob:=False:C215 + + If ($data#Null:C1517) + Case of + : (Value type:C1509($data)=Is object:K8:27) + $isFile:=OB Instance of:C1731($data; 4D:C1709.File) + $isBlob:=OB Instance of:C1731($data; 4D:C1709.Blob) + : (Value type:C1509($data)=Is BLOB:K8:12) + $isBlob:=True:C214 + End case + End if + + If (Not:C34($isFile) && Not:C34($isBlob)) + throw:C1805(1; "Expected a non-empty value for `data` (must be 4D.File or 4D.Blob/Blob)") + End if + + If (Not:C34(OB Instance of:C1731($parameters; cs:C1710.OpenAIParameters))) + $parameters:=cs:C1710.OpenAIParameters.new($parameters) + End if + + var $body : Object:=$parameters.body() + var $files : Object:={data: $data} + + return This:C1470._client._postFiles("/uploads/"+$uploadId+"/parts"; $body; $files; $parameters; cs:C1710.OpenAIUploadPartResult) + +/* +* Completes the Upload. +* +* Within the returned Upload object, there is a nested File object that is ready to use in the rest of the platform. +* You can specify the order of the Parts by passing in an ordered list of the Part IDs. +* The number of bytes uploaded upon completion must match the number of bytes initially specified when creating the Upload object. +* No Parts may be added after an Upload is completed. +* +* @param $uploadId {Text} The ID of the Upload (required) +* @param $part_ids {Collection} The ordered list of Part IDs (required) +* @param $parameters {cs.OpenAIUploadCompleteParameters} Optional parameters including md5 checksum +* @return {cs.OpenAIUploadResult} Result containing the Upload object with status completed and a file property +* @throws Error if uploadId is empty or part_ids is invalid +*/ +Function complete($uploadId : Text; $part_ids : Collection; $parameters : cs:C1710.OpenAIUploadCompleteParameters) : cs:C1710.OpenAIUploadResult + If (Length:C16($uploadId)=0) + throw:C1805(1; "Expected a non-empty value for `uploadId`") + End if + + If ($part_ids=Null:C1517) || ($part_ids.length=0) + throw:C1805(1; "Expected a non-empty collection for `part_ids`") + End if + + If (Not:C34(OB Instance of:C1731($parameters; cs:C1710.OpenAIUploadCompleteParameters))) + $parameters:=cs:C1710.OpenAIUploadCompleteParameters.new($parameters) + End if + + // Set required parameter + $parameters.part_ids:=$part_ids + + var $body : Object:=$parameters.body() + + return This:C1470._client._post("/uploads/"+$uploadId+"/complete"; $body; $parameters; cs:C1710.OpenAIUploadResult) + +/* +* Cancels the Upload. No Parts may be added after an Upload is cancelled. +* +* @param $uploadId {Text} The ID of the Upload (required) +* @param $parameters {cs.OpenAIParameters} Optional parameters for the request +* @return {cs.OpenAIUploadResult} Result containing the Upload object with status cancelled +* @throws Error if uploadId is empty +*/ +Function cancel($uploadId : Text; $parameters : cs:C1710.OpenAIParameters) : cs:C1710.OpenAIUploadResult + If (Length:C16($uploadId)=0) + throw:C1805(1; "Expected a non-empty value for `uploadId`") + End if + + If (Not:C34(OB Instance of:C1731($parameters; cs:C1710.OpenAIParameters))) + $parameters:=cs:C1710.OpenAIParameters.new($parameters) + End if + + var $body : Object:={} // Empty body for cancel request + + return This:C1470._client._post("/uploads/"+$uploadId+"/cancel"; $body; $parameters; cs:C1710.OpenAIUploadResult) + \ No newline at end of file diff --git a/Project/Sources/Methods/test_openai_uploads.4dm b/Project/Sources/Methods/test_openai_uploads.4dm new file mode 100644 index 0000000..3e1396c --- /dev/null +++ b/Project/Sources/Methods/test_openai_uploads.4dm @@ -0,0 +1,206 @@ +//%attributes = {"invisible":true} +var $client:=TestOpenAI() +If ($client=Null:C1517) + return // skip test +End if + +// MARK:- Setup: Create test data for multipart upload +var $testDataFolder:=Folder:C1567(Temporary folder:C486; fk platform path:K87:2).folder("OpenAI_Upload_Test") +If (Not:C34($testDataFolder.exists)) + $testDataFolder.create() +End if + +// Create test content (simulating a large file split into parts) +var $totalSize:=0 +var $chunks:=New collection:C1472 +var $chunkFiles:=New collection:C1472 + +// Create 3 test chunks (simulating a file split into parts) +var $i; $j : Integer +For ($i; 1; 3) + var $chunkFile:=$testDataFolder.file("chunk_"+String:C10($i)+".jsonl") + var $chunkContent:="" + + // Add multiple lines to each chunk + For ($j; 1; 5) + $chunkContent:=$chunkContent+"{\"messages\": [{\"role\": \"system\", \"content\": \"Test chunk "+String:C10($i)+" line "+String:C10($j)+"\"}, {\"role\": \"user\", \"content\": \"Sample question "+String:C10($j)+"\"}, {\"role\": \"assistant\", \"content\": \"Sample answer "+String:C10($j)+"\"}]}"+Char:C90(Line feed:K15:40) + End for + + $chunkFile.setText($chunkContent) + $chunkFiles.push($chunkFile) + $totalSize:=$totalSize+$chunkFile.size +End for + +var $uploadId:="" +var $partIds:=New collection:C1472 + +// MARK:- Test 1: Create upload +var $uploadParams:=cs:C1710.OpenAIUploadParameters.new() + +var $createResult:=$client.uploads.create("test_multipart.jsonl"; $totalSize; "fine-tune"; "text/jsonl"; $uploadParams) + +If (Asserted:C1132(Bool:C1537($createResult.success); "Cannot create upload: "+JSON Stringify:C1217($createResult))) + + If (Asserted:C1132($createResult.upload#Null:C1517; "Upload must not be null")) + + ASSERT:C1129(Length:C16(String:C10($createResult.upload.id))>0; "Upload ID must not be empty") + ASSERT:C1129(String:C10($createResult.upload.object)="upload"; "Object type must be 'upload'") + ASSERT:C1129(String:C10($createResult.upload.status)="pending"; "Initial status must be 'pending'") + ASSERT:C1129(String:C10($createResult.upload.filename)="test_multipart.jsonl"; "Filename must match") + ASSERT:C1129(String:C10($createResult.upload.purpose)="fine-tune"; "Purpose must match") + ASSERT:C1129($createResult.upload.bytes=$totalSize; "Bytes must match total size") + // ASSERT(String($createResult.upload.mime_type)="text/jsonl"; "MIME type must match") // seems not in response... + ASSERT:C1129((Value type:C1509($createResult.upload.created_at)=Is real:K8:4) && $createResult.upload.created_at>0; "Created timestamp must be set") + ASSERT:C1129((Value type:C1509($createResult.upload.expires_at)=Is real:K8:4) && $createResult.upload.expires_at>0; "Expires timestamp must be set") + + $uploadId:=$createResult.upload.id + + End if + +End if + +// MARK:- Test 2: Add parts to upload +If (Length:C16($uploadId)>0) + + For each ($chunkFile; $chunkFiles) + var $partParams:=cs:C1710.OpenAIParameters.new() + var $partResult:=$client.uploads.addPart($uploadId; $chunkFile; $partParams) + + If (Asserted:C1132(Bool:C1537($partResult.success); "Cannot add part: "+JSON Stringify:C1217($partResult))) + + If (Asserted:C1132($partResult.part#Null:C1517; "Part must not be null")) + + ASSERT:C1129(Length:C16(String:C10($partResult.part.id))>0; "Part ID must not be empty") + ASSERT:C1129(String:C10($partResult.part.object)="upload.part"; "Object type must be 'upload.part'") + ASSERT:C1129(String:C10($partResult.part.upload_id)=$uploadId; "Part upload_id must match upload ID") + ASSERT:C1129((Value type:C1509($partResult.part.created_at)=Is real:K8:4) && $partResult.part.created_at>0; "Part created timestamp must be set") + + $partIds.push($partResult.part.id) + + End if + + End if + End for each + +End if + +// MARK:- Test 3: Complete upload +If (Length:C16($uploadId)>0) && ($partIds.length=3) + + var $completeParams:=cs:C1710.OpenAIUploadCompleteParameters.new() + + var $completeResult:=$client.uploads.complete($uploadId; $partIds; $completeParams) + + If (Asserted:C1132(Bool:C1537($completeResult.success); "Cannot complete upload: "+JSON Stringify:C1217($completeResult))) + + If (Asserted:C1132($completeResult.upload#Null:C1517; "Completed upload must not be null")) + + ASSERT:C1129(String:C10($completeResult.upload.id)=$uploadId; "Completed upload ID must match") + ASSERT:C1129(String:C10($completeResult.upload.status)="completed"; "Status must be 'completed'") + + // Check if file object is present + If (Asserted:C1132($completeResult.upload.file#Null:C1517; "File object must be present after completion")) + + ASSERT:C1129(Length:C16(String:C10($completeResult.upload.file.id))>0; "File ID must not be empty") + ASSERT:C1129(String:C10($completeResult.upload.file.object)="file"; "File object type must be 'file'") + ASSERT:C1129(String:C10($completeResult.upload.file.filename)="test_multipart.jsonl"; "File filename must match") + ASSERT:C1129(String:C10($completeResult.upload.file.purpose)="fine-tune"; "File purpose must match") + ASSERT:C1129($completeResult.upload.file.bytes>0; "File bytes must be greater than 0") + + // Clean up: delete the created file + var $deleteResult:=$client.files.delete($completeResult.upload.file.id) + ASSERT:C1129(Bool:C1537($deleteResult.success); "Should be able to delete the created file") + + End if + + End if + + End if + +End if + +// MARK:- Test 4: Create and cancel upload +var $cancelParams:=cs:C1710.OpenAIUploadParameters.new() + +var $cancelCreateResult:=$client.uploads.create("test_cancel.jsonl"; 1000; "fine-tune"; "text/jsonl"; $cancelParams) + +If (Asserted:C1132(Bool:C1537($cancelCreateResult.success); "Cannot create upload for cancel test: "+JSON Stringify:C1217($cancelCreateResult))) + + If (Asserted:C1132($cancelCreateResult.upload#Null:C1517; "Upload for cancel must not be null")) + + var $cancelUploadId:=$cancelCreateResult.upload.id + + // Cancel the upload + var $cancelResult:=$client.uploads.cancel($cancelUploadId; cs:C1710.OpenAIParameters.new()) + + If (Asserted:C1132(Bool:C1537($cancelResult.success); "Cannot cancel upload: "+JSON Stringify:C1217($cancelResult))) + + If (Asserted:C1132($cancelResult.upload#Null:C1517; "Cancelled upload must not be null")) + + ASSERT:C1129(String:C10($cancelResult.upload.id)=$cancelUploadId; "Cancelled upload ID must match") + ASSERT:C1129(String:C10($cancelResult.upload.status)="cancelled"; "Status must be 'cancelled'") + + End if + + End if + + End if + +End if + +// MARK:- Test 5: Upload with Blob instead of File +var $blobUploadParams:=cs:C1710.OpenAIUploadParameters.new() + +var $blobCreateResult:=$client.uploads.create("test_blob.jsonl"; 100; "fine-tune"; "text/jsonl"; $blobUploadParams) + +If (Asserted:C1132(Bool:C1537($blobCreateResult.success); "Cannot create upload for blob test: "+JSON Stringify:C1217($blobCreateResult))) + + If (Asserted:C1132($blobCreateResult.upload#Null:C1517; "Blob upload must not be null")) + + var $blobUploadId:=$blobCreateResult.upload.id + + // Create a blob + var $testBlob : Blob + TEXT TO BLOB:C554("{\"messages\": [{\"role\": \"system\", \"content\": \"Test\"}]}"; $testBlob; UTF8 text without length:K22:17) + + // Add blob as part + var $blobPartResult:=$client.uploads.addPart($blobUploadId; $testBlob; cs:C1710.OpenAIParameters.new()) + + If (Asserted:C1132(Bool:C1537($blobPartResult.success); "Cannot add blob part: "+JSON Stringify:C1217($blobPartResult))) + + If (Asserted:C1132($blobPartResult.part#Null:C1517; "Blob part must not be null")) + + ASSERT:C1129(Length:C16(String:C10($blobPartResult.part.id))>0; "Blob part ID must not be empty") + ASSERT:C1129(String:C10($blobPartResult.part.upload_id)=$blobUploadId; "Blob part upload_id must match") + + // Cancel this test upload (don't complete it) + $client.uploads.cancel($blobUploadId; cs:C1710.OpenAIParameters.new()) + + End if + + End if + + End if + +End if + +// MARK:- Test 6: Error handling - Invalid upload ID +var $invalidPartResult:=$client.uploads.addPart("invalid_upload_id"; $chunkFiles[0]; cs:C1710.OpenAIParameters.new()) +ASSERT:C1129(Not:C34(Bool:C1537($invalidPartResult.success)); "Adding part to invalid upload ID should fail") + +// MARK:- Test 7: Error handling - Complete with wrong part IDs +var $testUpload2:=$client.uploads.create("test_error.jsonl"; 1000; "fine-tune"; "text/jsonl"; cs:C1710.OpenAIUploadParameters.new()) + +If (Bool:C1537($testUpload2.success)) + var $wrongCompleteParams:=cs:C1710.OpenAIUploadCompleteParameters.new() + + var $wrongPartIds:=New collection:C1472("invalid_part_1"; "invalid_part_2") + var $wrongCompleteResult:=$client.uploads.complete($testUpload2.upload.id; $wrongPartIds; $wrongCompleteParams) + ASSERT:C1129(Not:C34(Bool:C1537($wrongCompleteResult.success)); "Completing with invalid part IDs should fail") + + // Clean up + $client.uploads.cancel($testUpload2.upload.id; cs:C1710.OpenAIParameters.new()) +End if + +// MARK:- Cleanup: Remove test folder +$testDataFolder.delete(Delete with contents:K24:24) diff --git a/Project/Sources/Methods/test_openai_uploads_async.4dm b/Project/Sources/Methods/test_openai_uploads_async.4dm new file mode 100644 index 0000000..9a97971 --- /dev/null +++ b/Project/Sources/Methods/test_openai_uploads_async.4dm @@ -0,0 +1,212 @@ +//%attributes = {"invisible":true} +var $client:=TestOpenAI() +If ($client=Null:C1517) + return // skip test +End if + +// MARK:- Setup: Create test data +var $testDataFolder:=Folder:C1567(Temporary folder:C486; fk platform path:K87:2).folder("OpenAI_Upload_Test_Async") +If (Not:C34($testDataFolder.exists)) + $testDataFolder.create() +End if + +// Create test chunks +var $totalSize:=0 +var $chunkFiles:=New collection:C1472 + +var $i; $j : Integer +For ($i; 1; 3) + var $chunkFile:=$testDataFolder.file("async_chunk_"+String:C10($i)+".jsonl") + var $chunkContent:="" + + For ($j; 1; 5) + $chunkContent:=$chunkContent+"{\"messages\": [{\"role\": \"system\", \"content\": \"Async test chunk "+String:C10($i)+" line "+String:C10($j)+"\"}, {\"role\": \"user\", \"content\": \"Q"+String:C10($j)+"\"}, {\"role\": \"assistant\", \"content\": \"A"+String:C10($j)+"\"}]}"+Char:C90(Line feed:K15:40) + End for + + $chunkFile.setText($chunkContent) + $chunkFiles.push($chunkFile) + $totalSize:=$totalSize+$chunkFile.size +End for + +var $uploadId:="" +var $partIds:=New collection:C1472 + +// MARK:- Test 1: Create upload (async) +cs:C1710._TestSignal.me.init() + +CALL WORKER:C1389(Current method name:C684; Formula:C1597($client.uploads.create("test_async_multipart.jsonl"; $totalSize; "fine-tune"; "text/jsonl"; {formula: Formula:C1597(cs:C1710._TestSignal.me.trigger($1))}))) + +cs:C1710._TestSignal.me.wait(15*1000) + +var $createResult : cs:C1710.OpenAIUploadResult:=cs:C1710._TestSignal.me.result + +If (Asserted:C1132(Bool:C1537($createResult.success); "Cannot create upload (async): "+JSON Stringify:C1217($createResult))) + + If (Asserted:C1132($createResult.upload#Null:C1517; "Async upload must not be null")) + + ASSERT:C1129(Length:C16(String:C10($createResult.upload.id))>0; "Async upload ID must not be empty") + ASSERT:C1129(String:C10($createResult.upload.object)="upload"; "Object type must be 'upload'") + ASSERT:C1129(String:C10($createResult.upload.status)="pending"; "Initial status must be 'pending'") + ASSERT:C1129(String:C10($createResult.upload.filename)="test_async_multipart.jsonl"; "Filename must match") + ASSERT:C1129(String:C10($createResult.upload.purpose)="fine-tune"; "Purpose must match") + ASSERT:C1129($createResult.upload.bytes=$totalSize; "Bytes must match") + + $uploadId:=$createResult.upload.id + + End if + +End if + +cs:C1710._TestSignal.me.reset() + +// MARK:- Test 2: Add parts (async) +If (Length:C16($uploadId)>0) + + For each ($chunkFile; $chunkFiles) + cs:C1710._TestSignal.me.init() + + CALL WORKER:C1389(Current method name:C684; Formula:C1597($client.uploads.addPart($uploadId; $chunkFile; {formula: Formula:C1597(cs:C1710._TestSignal.me.trigger($1))}))) + + cs:C1710._TestSignal.me.wait(15*1000) + + var $partResult : cs:C1710.OpenAIUploadPartResult:=cs:C1710._TestSignal.me.result + + If (Asserted:C1132(Bool:C1537($partResult.success); "Cannot add part (async): "+JSON Stringify:C1217($partResult))) + + If (Asserted:C1132($partResult.part#Null:C1517; "Async part must not be null")) + + ASSERT:C1129(Length:C16(String:C10($partResult.part.id))>0; "Async part ID must not be empty") + ASSERT:C1129(String:C10($partResult.part.object)="upload.part"; "Object type must be 'upload.part'") + ASSERT:C1129(String:C10($partResult.part.upload_id)=$uploadId; "Part upload_id must match") + + $partIds.push($partResult.part.id) + + End if + + End if + + cs:C1710._TestSignal.me.reset() + + End for each + +End if + +// MARK:- Test 3: Complete upload (async) +If (Length:C16($uploadId)>0) && ($partIds.length=3) + + cs:C1710._TestSignal.me.init() + + CALL WORKER:C1389(Current method name:C684; Formula:C1597($client.uploads.complete($uploadId; $partIds; {formula: Formula:C1597(cs:C1710._TestSignal.me.trigger($1))}))) + + cs:C1710._TestSignal.me.wait(15*1000) + + var $completeResult : cs:C1710.OpenAIUploadResult:=cs:C1710._TestSignal.me.result + + If (Asserted:C1132(Bool:C1537($completeResult.success); "Cannot complete upload (async): "+JSON Stringify:C1217($completeResult))) + + If (Asserted:C1132($completeResult.upload#Null:C1517; "Async completed upload must not be null")) + + ASSERT:C1129(String:C10($completeResult.upload.id)=$uploadId; "Async completed upload ID must match") + ASSERT:C1129(String:C10($completeResult.upload.status)="completed"; "Status must be 'completed'") + + If (Asserted:C1132($completeResult.upload.file#Null:C1517; "File object must be present")) + + ASSERT:C1129(Length:C16(String:C10($completeResult.upload.file.id))>0; "File ID must not be empty") + ASSERT:C1129(String:C10($completeResult.upload.file.object)="file"; "File object type must be 'file'") + ASSERT:C1129(String:C10($completeResult.upload.file.filename)="test_async_multipart.jsonl"; "Filename must match") + + // Clean up: delete the created file (async) + cs:C1710._TestSignal.me.reset() + cs:C1710._TestSignal.me.init() + + CALL WORKER:C1389(Current method name:C684; Formula:C1597($client.files.delete($completeResult.upload.file.id; {formula: Formula:C1597(cs:C1710._TestSignal.me.trigger($1))}))) + + cs:C1710._TestSignal.me.wait(10*1000) + + var $deleteResult : cs:C1710.OpenAIFileDeletedResult:=cs:C1710._TestSignal.me.result + ASSERT:C1129(Bool:C1537($deleteResult.success); "Should be able to delete the created file (async)") + + End if + + End if + + End if + + cs:C1710._TestSignal.me.reset() + +End if + +// MARK:- Test 4: Cancel upload (async) +cs:C1710._TestSignal.me.init() + +CALL WORKER:C1389(Current method name:C684; Formula:C1597($client.uploads.create("test_async_cancel.jsonl"; 1000; "fine-tune"; "text/jsonl"; {formula: Formula:C1597(cs:C1710._TestSignal.me.trigger($1))}))) + +cs:C1710._TestSignal.me.wait(15*1000) + +var $cancelCreateResult : cs:C1710.OpenAIUploadResult:=cs:C1710._TestSignal.me.result + +If (Asserted:C1132(Bool:C1537($cancelCreateResult.success); "Cannot create upload for async cancel test: "+JSON Stringify:C1217($cancelCreateResult))) + + If (Asserted:C1132($cancelCreateResult.upload#Null:C1517; "Upload for async cancel must not be null")) + + var $cancelUploadId:=$cancelCreateResult.upload.id + + cs:C1710._TestSignal.me.reset() + cs:C1710._TestSignal.me.init() + + // Cancel the upload + CALL WORKER:C1389(Current method name:C684; Formula:C1597($client.uploads.cancel($cancelUploadId; {formula: Formula:C1597(cs:C1710._TestSignal.me.trigger($1))}))) + + cs:C1710._TestSignal.me.wait(15*1000) + + var $cancelResult : cs:C1710.OpenAIUploadResult:=cs:C1710._TestSignal.me.result + + If (Asserted:C1132(Bool:C1537($cancelResult.success); "Cannot cancel upload (async): "+JSON Stringify:C1217($cancelResult))) + + If (Asserted:C1132($cancelResult.upload#Null:C1517; "Cancelled upload must not be null")) + + ASSERT:C1129(String:C10($cancelResult.upload.id)=$cancelUploadId; "Async cancelled upload ID must match") + ASSERT:C1129(String:C10($cancelResult.upload.status)="cancelled"; "Status must be 'cancelled'") + + End if + + End if + + End if + +End if + +cs:C1710._TestSignal.me.reset() + +// MARK:- Test 5: Using onResponse/onError callbacks +cs:C1710._TestSignal.me.init() + +CALL WORKER:C1389(Current method name:C684; Formula:C1597($client.uploads.create("test_callback.jsonl"; 500; "fine-tune"; "text/jsonl"; {\ +onResponse: Formula:C1597(cs:C1710._TestSignal.me.trigger($1)); \ +onError: Formula:C1597(cs:C1710._TestSignal.me.trigger($1))\ +}))) + +cs:C1710._TestSignal.me.wait(15*1000) + +var $callbackResult : cs:C1710.OpenAIUploadResult:=cs:C1710._TestSignal.me.result + +If (Asserted:C1132(Bool:C1537($callbackResult.success); "Cannot create upload with callbacks (async): "+JSON Stringify:C1217($callbackResult))) + + If (Asserted:C1132($callbackResult.upload#Null:C1517; "Callback upload must not be null")) + + ASSERT:C1129(Length:C16(String:C10($callbackResult.upload.id))>0; "Callback upload ID must not be empty") + ASSERT:C1129(String:C10($callbackResult.upload.status)="pending"; "Callback upload status must be 'pending'") + + // Cancel this test upload + $client.uploads.cancel($callbackResult.upload.id; cs:C1710.OpenAIParameters.new()) + + End if + +End if + +cs:C1710._TestSignal.me.reset() + +// MARK:- Cleanup: Remove test folder +$testDataFolder.delete(Delete with contents:K24:24) + +KILL WORKER:C1390(Current method name:C684) diff --git a/Project/Sources/folders.json b/Project/Sources/folders.json index c551893..3a72f59 100644 --- a/Project/Sources/folders.json +++ b/Project/Sources/folders.json @@ -116,6 +116,8 @@ "test_openai_moderations", "test_openai_msg", "test_openai_msg_accumulate", + "test_openai_uploads", + "test_openai_uploads_async", "test_openai_vision", "TestOpenAI" ], @@ -124,6 +126,17 @@ "_TestSignal" ] }, + "Uploads": { + "classes": [ + "OpenAIUpload", + "OpenAIUploadCompleteParameters", + "OpenAIUploadParameters", + "OpenAIUploadPart", + "OpenAIUploadPartResult", + "OpenAIUploadResult", + "OpenAIUploadsAPI" + ] + }, "Utils": { "methods": [ "_autoClassDoc" diff --git a/README.md b/README.md index 4909646..c594866 100644 --- a/README.md +++ b/README.md @@ -130,6 +130,39 @@ Delete a file var $deleteResult:=$client.files.delete($fileId) ``` +#### Uploads + +https://platform.openai.com/docs/api-reference/uploads + +Upload large files (up to 8 GB) in multiple parts + +```4d +var $partIds:=New collection + +// Step 1: Create upload +var $params:=cs.AIKit.OpenAIUploadParameters.new() +$params.filename:="training_data.jsonl" +$params.purpose:="fine-tune" +$params.bytes:=200000000 // File size in bytes +$params.mime_type:="text/jsonl" + +var $result:=$client.uploads.create($params) +var $uploadId:=$result.upload.id + +// Step 2: Add parts (up to 64 MB each) +For ($i; 1; 4) + var $file:=File("/path/to/chunk_"+String($i)+".dat") + var $partResult:=$client.uploads.addPart($uploadId; $file; cs.AIKit.OpenAIParameters.new()) + $partIds.push($partResult.part.id) +End for + +// Step 3: Complete upload +var $completeParams:=cs.AIKit.OpenAIUploadCompleteParameters.new() +$completeParams.part_ids:=$partIds +var $completeResult:=$client.uploads.complete($uploadId; $completeParams) +var $fileId:=$completeResult.upload.file.id +``` + #### Moderations https://platform.openai.com/docs/api-reference/moderations