Add instructions to run OpenSearch from source code

[Yerzhaisang]

Description

While working on a recent issue, I faced challenges figuring out how to test my implementation locally. Thankfully, @brianf-aws guided me on how to run ml-commons from the source code.

To help new contributors avoid similar confusion, I’ve added clear instructions on how to build and run OpenSearch from source, enabling them to test their changes effectively.

Related Issues

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[dhrubo-os]

Thanks a lot for raising this PR; it's something I've been planning to raise for a while! :D

[brianf-aws]

Thanks for taking the initiative! Left some comments to enhance the tutorial

[brianf-aws]

We can also explain what this is doing for example you can create a OS cluster (2 data nodes 2 ml nodes) running your custom Ml-Commons code. We should also imply that this does not run OSD (OS dashboards)

[dhrubo-os]

May be we can add this too so that OSD can be added to the setup too [We might need to add : opensearchstaging/opensearch:3.0.0]:

opensearch-dashboards:
    image: opensearchstaging/opensearch-dashboards:latest
    container_name: opensearch-dashboards
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
    networks:
      - opensearch-net

[brianf-aws]

@Yerzhaisang Be careful when adding the dashboards part. You would need to set the security plugin on (it turns on https between nodes.)

      - "DISABLE_SECURITY_PLUGIN=false"

or omitting it should help you use https

[Yerzhaisang]

hey @dhrubo-os , should these lines work?
When I open dashboard, I see only OpenSearch Dashboards server is not ready yet

[brianf-aws]

@Yerzhaisang This part is tricky; when you do dashboards it expects https nodes check your OS nodes to see that security plugin is turned on.

[Yerzhaisang]

so does developer really need dashboards or is it possible to handle anything with curl and apps like Postman ?

[brianf-aws]

Good question, When I need to test something involving a system index I cant use dashboards as it requires https traffic which means the security plugin needs to be turned on. This is just my dev experience I dont have these options down to a science. Also from my experience its hard to debug when you have dashboards enabled it tends to time out (there is some setting to extend this time for OSD to wait for a response).

If your testing usually doesnt involve using the security plugin removed then dashboards is nice to use.

[Yerzhaisang]

@dhrubo-os could you please share your opinion? I think a developer doesn't need OSD (my opinion)

[ylwu-amzn]

How about move the docker compose file to docs/docker ?

[dhrubo-os]

@Yerzhaisang take updates from upstream for integ tests to pass.