Skip to content

Commit bf6f24c

Browse files
DataUI VCS Robottatlush
DataUI VCS Robot
authored andcommitted
Release 10.02.2025
* Compute Cloud: added instructions on how to manage disk encryption via YC CLI. * Managed Service for Kubernetes: updated k8s versions. * Object Storage: added a tutorial on how to store Veeam backups in a bucket. * Smart Web Security: added a tutorial on how to configure Smart protection for backends in Manage Service for Kubernetes cluster via Application Load Balancer Ingress controller. * Translations updated. * Fixes and improvements.
1 parent 1d30ff5 commit bf6f24c

File tree

214 files changed

+5437
-1238
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

214 files changed

+5437
-1238
lines changed

en/_includes/datasphere/migration/configurations.md

+9-9
Original file line numberDiff line numberDiff line change
@@ -5,25 +5,25 @@ Since projects run on different {{ yandex-cloud }} platforms with different comp
55
{% endnote %}
66

77

8-
| Name | Number of vCPUs | Number of GPUs | RAM, GB | VRAM, GB |
8+
Name | Number of vCPUs | Number of GPUs | RAM, GB | VRAM, GB
99
---- | ---- | ---- | ---- | ----
1010
**Configurations with vCPUs based on [Intel Ice Lake](../../../compute/concepts/performance-levels.md)**
1111
**c1.4** (default) | 4 | 0 | 32 | —
1212
**c1.8** | 8 | 0 | 64 | —
1313
**c1.32** ^1^ | 32 | 0 | 256 | —
1414
**c1.80** ^2^ | 80 | 0 | 640 | —
15-
**Configurations with vCPUs based on [Intel Broadwell](../../../compute/concepts/performance-levels.md) and [NVIDIA® Tesla® V100](../../../compute/concepts/gpus.md) GPU**
16-
**g1.1** | 8 | 1 | From 48 to 96 | 32
17-
**g1.2** | 16 | 2 | From 96 to 192 | 64
18-
**g1.4** ^1^ | 32 | 4 | From 192 to 384 | 128
19-
**Configurations with vCPUs based on [AMD EPYC™](../../../compute/concepts/gpus.md) and [NVIDIA® Ampere® A100](https://www.nvidia.com/en-us/data-center/a100/) GPU**
15+
**Configurations with [Intel Broadwell](../../../compute/concepts/performance-levels.md) vCPUs and [NVIDIA® Tesla® V100](../../../compute/concepts/gpus.md) GPUs**
16+
**g1.1** | 8 | 1 | 48 to 96 | 32
17+
**g1.2** | 16 | 2 | 96 to 192 | 64
18+
**g1.4** ^1^ | 32 | 4 | 192 to 384 | 128
19+
**Configurations with [AMD EPYC™](../../../compute/concepts/gpus.md) vCPUs and [NVIDIA® Ampere® A100](https://www.nvidia.com/en-us/data-center/a100/) GPUs**
2020
**g2.1** ^1^ | 28 | 1 | 119 | 80
2121
**g2.2** ^1^ | 56 | 2 | 238 | 160
2222
**g2.4** ^1^ | 112 | 4 | 476 | 320
23-
**g2.8** ^2^ | 224 | 8 | 952 | From 320 to 640
24-
**Configurations with vCPUs based on [Intel Ice Lake](../../../compute/concepts/performance-levels.md) and [NVIDIA® Tesla® T4](https://www.nvidia.com/en-us/data-center/tesla-t4/) GPU**
23+
**g2.8** ^2^ | 224 | 8 | 952 | 320 to 640
24+
**Configurations with [Intel Ice Lake](../../../compute/concepts/performance-levels.md) vCPUs and [NVIDIA® Tesla® T4](https://www.nvidia.com/en-us/data-center/tesla-t4/) GPUs**
2525
**gt4.1** ^1^ | 4 | 1 | 16 | 16
26-
**Configurations with vCPUs based on [Intel Ice Lake](../../../compute/concepts/performance-levels.md) and [NVIDIA® Tesla® T4i](../../../compute/concepts/gpus.md#t4i) GPU**
26+
**Configurations with [Intel Ice Lake](../../../compute/concepts/performance-levels.md) vCPUs and [NVIDIA® Tesla® T4i](../../../compute/concepts/gpus.md#t4i) GPUs**
2727
**gt4i.1** ^1^ | 8 | 1 | 32 | 24
2828

2929
^1^ Available after you deposit at least $10 to your billing account or upon request to the support team.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
```python
2+
#!/usr/bin/env python3
3+
4+
from __future__ import annotations
5+
import asyncio
6+
import pathlib
7+
from yandex_cloud_ml_sdk import AsyncYCloudML
8+
from yandex_cloud_ml_sdk.auth import YandexCloudCLIAuth
9+
10+
11+
def local_path(path: str) -> pathlib.Path:
12+
return pathlib.Path(__file__).parent / path
13+
14+
15+
async def main():
16+
17+
sdk = AsyncYCloudML(
18+
folder_id="<folder_ID>",
19+
auth="<API_key>",
20+
)
21+
22+
# Creating a tuning dataset for the {{ gpt-lite }} base model
23+
dataset_draft = sdk.datasets.draft_from_path(
24+
task_type="<classification_type>",
25+
path="<path_to_file>",
26+
upload_format="jsonlines",
27+
name="multiclass",
28+
)
29+
30+
# Waiting for the data to be uploaded and the dataset to be created
31+
operation = await dataset_draft.upload_deferred()
32+
dataset = await operation
33+
print(f"new {dataset=}")
34+
35+
36+
if __name__ == "__main__":
37+
asyncio.run(main())
38+
```
39+
40+
Where:
41+
42+
* `<folder_ID>`: [ID of the folder](../../../resource-manager/operations/folder/get-id.md) the [service account](../../../iam/concepts/users/service-accounts.md) was created in.
43+
* `<API_key>`: Service account [API key](../../../iam/concepts/authorization/api-key.md) you got earlier required for [authentication in the API](../../../foundation-models/api-ref/authentication.md).
44+
45+
{% include [sdk-auth-details-paragraph](../sdk-auth-details-paragraph.md) %}
46+
* `<classification_type>`: [Classification type](../../../foundation-models/concepts/classifier/index.md) the model will be tuned for using the new dataset. The possible values are:
47+
48+
* `TextClassificationMultilabel`: Binary classification or multi-label classification.
49+
* `TextClassificationMulticlass`: Multi-class classification.
50+
* `<file_path>`: Path to the file containing the ready-made examples for model tuning.

en/_includes/foundation-models/examples/dataset-sdk.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,15 @@ async def main():
2020
)
2121

2222
# Creating a tuning dataset for the {{ gpt-lite }} base model
23-
dataset_draft = sdk.datasets.from_path_deferred(
23+
dataset_draft = sdk.datasets.draft_from_path(
2424
task_type="TextToTextGeneration",
2525
path="<path_to_file>",
2626
upload_format="jsonlines",
2727
name="YandexGPT tuning",
2828
)
2929

3030
# Waiting for the data to be uploaded and the dataset to be created
31-
operation = await dataset_draft.upload()
31+
operation = await dataset_draft.upload_deferred()
3232
dataset = await operation
3333
print(f"new {dataset=}")
3434

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
```python
2+
#!/usr/bin/env python3
3+
4+
from __future__ import annotations
5+
import pathlib
6+
import uuid
7+
from yandex_cloud_ml_sdk import YCloudML
8+
9+
10+
def local_path(path: str) -> pathlib.Path:
11+
return pathlib.Path(__file__).parent / path
12+
13+
14+
def main():
15+
sdk = YCloudML(
16+
folder_id="<folder_ID>",
17+
auth="<API_key>",
18+
)
19+
20+
# Viewing the list of valid datasets
21+
for dataset in sdk.datasets.list(status="READY", name_pattern="multiclass"):
22+
print(f"List of existing datasets {dataset=}")
23+
24+
# Setting the tuning dataset and the base model
25+
train_dataset = sdk.datasets.get("<dataset_ID>")
26+
base_model = sdk.models.text_classifiers("yandexgpt-lite")
27+
28+
# Defining minimum parameters
29+
# To control more parameters, use `base_model.tune_deferred()`
30+
tuned_model = base_model.tune(
31+
train_dataset, name=str(uuid.uuid4()), classification_type="<classification_type>"
32+
)
33+
print(f"Resulting {tuned_model}")
34+
35+
# Starting the tuning
36+
classification_result = tuned_model.run("hey!")
37+
print(f"{classification_result=}")
38+
39+
# Saving the URI of the tuned model
40+
tuned_uri = tuned_model.uri
41+
model = sdk.models.text_classifiers(tuned_uri)
42+
43+
classification_result = model.run("hey!")
44+
print(f"{classification_result=}")
45+
46+
47+
if __name__ == "__main__":
48+
main()
49+
```
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
_Model tuning based on the {{ lora }} method is at the [Preview](../../overview/concepts/launch-stages.md) stage and available upon request. You can fill out the form in the [management console]({{ link-console-main }}/link/foundation-models/)._

en/_includes/foundation-models/public-talks.md

+5-1
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@
33

44
## Videos {#video}
55

6+
### 2025 {#2025}
7+
8+
* {{ yandexart-name }} for businesses. Watch the video on [YouTube](https://www.youtube.com/watch?v=I9Fpsxv8Hlc) or [Yandex](https://runtime.strm.yandex.ru/player/episode/vplefdetiqv7tasxrbyk).
9+
610
### 2024 {#2024}
711

812
* Creating AI assistants. [Click here to view the video](https://www.youtube.com/watch?v=kKbMaWSi20I).
@@ -37,7 +41,7 @@
3741

3842
### 2023 {#2023-articles}
3943

40-
* Feedback analysis, chat assistance, and creatives: what {{ yagpt-full-name }}'s skillset. [Read our article](https://vc.ru/services/945084-analiz-otzyvov-pomosh-v-chate-i-kreativy-chto-umeet-yandexgpt-api)
44+
* Feedback analysis, chat assistance, and creatives: {{ yagpt-full-name }}'s skillset. [Read our article](https://vc.ru/services/945084-analiz-otzyvov-pomosh-v-chate-i-kreativy-chto-umeet-yandexgpt-api)
4145

4246
* How to ditch servers, get your business analytics going, and give {{ yagpt-name }} a test. [Read our article](https://vc.ru/offline/845622-oboitis-bez-serverov-nastroit-biznes-analitiku-i-protestirovat-yandexgpt)
4347

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
To use the examples:
2+
3+
{% list tabs group=programming_language %}
4+
5+
- SDK {#sdk}
6+
7+
1. [Create](../../iam/operations/sa/create.md) a service account and [assign](../../iam/operations/sa/assign-role-for-sa.md) the `ai.editor` [role](../../foundation-models/security/index.md#languageModels-user) to it.
8+
1. [Get](../../iam/operations/api-key/create.md) the service account API key and save it.
9+
10+
{% include [sdk-auth-details-paragraph](./sdk-auth-details-paragraph.md) %}
11+
1. Use the [pip](https://pip.pypa.io/en/stable/) package manager to install the [{{ ml-sdk-name }} library](../../foundation-models/sdk/index.md):
12+
13+
```bash
14+
pip install yandex-cloud-ml-sdk
15+
```
16+
17+
- cURL {#curl}
18+
19+
1. {% include notitle [ai-before-beginning](./yandexgpt/ai-before-beginning.md) %}
20+
1. Install [gRPCurl](https://github.com/fullstorydev/grpcurl).
21+
1. {% include [curl](../curl.md) %}
22+
1. (Optional) Install the [jq](https://stedolan.github.io/jq/) JSON stream processor.
23+
1. [Get an IAM token](../../iam/operations/iam-token/create.md) used for authentication in the API.
24+
25+
{% note info %}
26+
27+
The IAM token has a short [lifetime](../../iam/concepts/authorization/iam-token.md#lifetime): no more than {{ iam-token-lifetime }}.
28+
29+
{% endnote %}
30+
31+
{% endlist %}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
2+
Get a link to upload data into the dataset:
3+
4+
```bash
5+
grpcurl \
6+
-H "Authorization: Bearer <IAM_token>" \
7+
-d '{"dataset_id": "<dataset_ID>", "size_bytes": <dataset_size>}' \
8+
{{ api-host-llm }}:443 yandex.cloud.ai.dataset.v1.DatasetService/GetUploadDraftUrl | jq
9+
```
10+
11+
Where:
12+
* `<IAM_token>`: [IAM token](../../iam/concepts/authorization/iam-token.md) of the service account you got [before you started](#before-begin).
13+
* `<dataset_ID>`: Dataset ID you saved in the previous step.
14+
* `<dataset_size>`: Size in bytes of the file with data for tuning. In the terminal, you can get the file size using the `ls -l <file_path>` command.
15+
16+
Result:
17+
18+
```text
19+
{
20+
"datasetId": "fdso08c1u1cq********",
21+
"uploadUrl": "https://storage.yandexcloud.net/ai-fomo-drafts-prod/b1gt6g8ht345********/fdso08c1u1cq********?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20250120T105352Z&X-Amz-SignedHeaders=content-length%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=YCAJE_WuJJ9D1r6huCoc8I3yO%2F20250120%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=611d7951994ae939acf4d32cc0c154c738d02adb2a04707a704f34ca********"
22+
}
23+
```
24+
25+
The `uploadUrl` field of the response contains a link you can use to upload your data into the dataset.
26+
27+
{% note tip %}
28+
29+
If you did not use jq, replace all `\u0026` occurrences with `&` in the link to use it to upload the dataset.
30+
31+
{% endnote %}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Upload your data by specifying the link you got in the previous step and the path to the fine-tuning data file:
2+
3+
```bash
4+
curl \
5+
--request PUT \
6+
--upload-file <path_to_file> \
7+
"<link>"
8+
```
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
After the data upload is complete, run the dataset validation:
2+
3+
```bash
4+
grpcurl \
5+
-H "Authorization: Bearer <IAM_token>" \
6+
-d '{"dataset_id": "<dataset_ID>"}' \
7+
{{ api-host-llm }}:443 yandex.cloud.ai.dataset.v1.DatasetService/Validate
8+
```
9+
10+
Where:
11+
* `<IAM_token>`: [IAM token](../../iam/concepts/authorization/iam-token.md) of the service account you got [before you started](#before-begin).
12+
* `<dataset_ID>`: Dataset ID you saved in the previous step.
13+
14+
Result:
15+
16+
```text
17+
{
18+
"id": "fdso01v2jdd4********",
19+
"createdAt": "2025-01-20T11:03:48Z",
20+
"modifiedAt": "2025-01-20T11:03:48Z"
21+
}
22+
```
23+
24+
Save the validation operation ID (`id` field). You will need it in the next step.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
Dataset validation may take some time. To find out validation status and get an error report (if any), send this request:
2+
3+
```bash
4+
grpcurl \
5+
-H "Authorization: Bearer <IAM_token>" \
6+
-d '{"operation_id": "<validation_operation_ID>"}' \
7+
{{ api-host-llm }}:443 yandex.cloud.operation.OperationService/Get
8+
```
9+
10+
Where:
11+
* `<IAM_token>`: [IAM token](../../iam/concepts/authorization/iam-token.md) of the service account you got [before you started](#before-begin).
12+
* `<validation_operation_ID>`: ID of the validation operation you saved in the previous step.
13+
14+
Result:
15+
16+
```text
17+
{
18+
"id": "fdso01v2jdd4********",
19+
"createdAt": "2025-01-20T11:03:48Z",
20+
"modifiedAt": "2025-01-20T11:04:46Z",
21+
"done": true,
22+
"response": {
23+
"@type": "type.googleapis.com/yandex.cloud.ai.dataset.v1.ValidateDatasetResponse",
24+
"datasetId": "fdso08c1u1cq********",
25+
"isValid": true
26+
}
27+
}
28+
```
29+
30+
The `isValid` field is set to `true`. This means the loaded dataset was validated successfully.

en/_includes/speechkit/async-modes.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
For [asynchronous recognition](../../{{ speechkit-slug }}/stt/transcribation.md), a language model that can operate in two modes is available:
22
1. In standard mode, recognition is processed in a standard priority queue. This mode works when the `general` model is selected.
3-
1. In deferred mode, the audio file to be recognized is placed in a low priority queue and processed at the least busy time. [Special pricing](../../{{ speechkit-slug }}/pricing.md#prices-stt) applies to deferred recognition. The time required to process an audio file in deferred mode is 24 hours or less. Recognition in deferred mode is available when the `deferred-general` tag is specified.
3+
1. In deferred mode, the audio file to be recognized is placed in a low priority queue and processed at the least busy time. [Special pricing](../../{{ speechkit-slug }}/pricing.md#prices-stt) applies to deferred recognition. The time required to process an audio file in deferred mode is 24 hours or less. Recognition in deferred mode is only available with [API v2](../../speechkit/stt/api/transcribation-api.md) when the `deferred-general` tag is specified.

en/_pricing_examples/datasphere/usd-inference.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
> 72 × 185,760 × 2 = 26,749,440 units for running a node
1+
> 72 × 185,760 × 2 = 26,749,440 units for the node
22
> 26,749,440 × $0.0000096 = $256.794624
33
>
4-
> Total: $256,794624, cost of using {{ ml-platform-name }}.
4+
> Total: $256.794624, cost of using {{ ml-platform-name }}.
55
66
Where:
77
* 72: Number of units per g1.1 configuration.

en/_qa/speechkit/stt.md

+1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# Speech recognition (STT)
22

3+
34
#### Incorrect stress and pronunciation {#stt-stress-pronunciation}
45

56
Create a request and attach examples so that developers can make adjustments to the next releases of the speech synthesis model.

0 commit comments

Comments
 (0)