v1.4.0

.github/workflows/release.yml

@@ -1,4 +1,4 @@
-name: release
+name: Automatic Release
 
 on:
   push:
@@ -19,8 +19,10 @@ jobs:
         lfs: true
         submodules: false
         fetch-depth: 0
-    - name: Set env
-      run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
+    - name: Set release version
+      run: |
+        echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV;
+        echo "${GITHUB_REF#refs/*/}" > VERSION
     - name: Compress services
       run: >
         tar -c --exclude-ignore=.tarignore -zvf /tmp/sh-services-${{ env.RELEASE_VERSION }}.tar.gz .
@@ -43,4 +45,4 @@ jobs:
         upload_url: ${{ steps.create_release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps 
         asset_path: /tmp/sh-services-${{ env.RELEASE_VERSION }}.tar.gz
         asset_name: sh-services-${{ env.RELEASE_VERSION }}.tar.gz
-        asset_content_type: application/tar+gzip
+        asset_content_type: application/tar+gzip

README.md

@@ -1,25 +1,56 @@
-# Sighthound docker-compose service
+# Sighthound Services
 
-## Quick start
+This repository hosts a collection of services for the SIO (Sighthound.IO) ecosystem. Services are intended to run via the `./scripts/sh-services` script, which relies on docker-compose. However, docker-compose is not strictly necessary for configuring or using this repository.
+
+The included services are as follow:
+- SIO: The computer vision analytics engine.
+- MCP: Media manager service, which includes a REST API and a cleaner. MCP relies on sharing media store folders with SIO service, and listens on AMQP message bus for media creation events, such as new video recording segments or event-driven jpeg images. It then provides the API access to that media (for documentation go to http://localhost:9097), and control its lifecycle.
+- RabbitMQ: Default AMQP broker and messaging platform.
+- Live555: An RTSP server for testing purposes. Note: This is disabled by default.
+- AMQP Stats: Connects to RabbitMQ and displays SIO output data. Note: This is disabled by default.
+
+This repository can serve either as a turnkey deployment model, or as a guide for creating customer-specific application deployments using SIO and MCP. For illustration of some sample uses, please refer to the examples folder.
 
-This repo contains a set of docker-compose services that are meant to be handled using the `./scripts/sh-services` script. It is a simple tool that calls `docker-compose up/down` but the main logic resides in the configuration management. The CLI tool reads the `conf` folder of every service and performs a merge (using alphanumeric priority) into the `.env` file that docker-compose uses.
+The `./scripts/sh-services` script is a basic tool that triggers `docker-compose up/down` commands as needed. However, the primary logic lies in configuration management. This Command-Line Interface (CLI) tool reads the `conf` folder for each service and merges the configurations, based on alphanumeric priority, into the `.env` file used by docker-compose.
 
+In the turnkey scenario, each service is managed with an individual `docker-compose` configuration file, an optional (or autogenerated by sh-serviecs) `.env` file containing relevant environment variable, and a collection of service specific configuration file in conf subfolder. To assist in orchestrating the services collection and disjointed `docker-compose` and environment configuration, `sh-services` CLI utility was introduced.
 
-### Node devices
+## Configuration Priority:
 
-If you are on a device that doesn't have `sh-device-ui` installed, you need to manually:
+For example, if you have the following  configuration files:
+ - default.env
+ - 0009-customer.env
+ - 0001-system.env
 
-- Install Sighthound Services
-- Install license and key files
-- Login to the Docker registry
+The priority of these configurations would be:
+ - 0001-system.env
+ - 0009-customer.env
+ - default.env
+
+The first file is given the highest priority, so it will overwrite any conflicting settings in files processed later.
+
+## Quick start
 
-#### Installing Sighthound Services
+This guide will help you set up SIO to point to a fake RTSP generated by live555 and start processing video.
+
+### Prerequisites (for non-dnncam devices)
+On Sighthound DNNCam devices, services come preinstalled, and the device GUI interacts with it. If you are using a dnncam we suggest you rely on the GUI to configure/update services, though it's not a requirement. On other devices, you need to manually:
+
+- 1: Install Sighthound Services
+- 2: Install license and key files
+- 3: Login to the Docker registry
+
+#### Prerequisit 1: Installing Sighthound Services
+
+Start by setting up a base directory for Sighthound:
 
 ```bash
-mkdir -p /data/sighthound
+SH_BASE="/data/sighthound"
+mkdir -p "${SH_BASE}"
+cd "${SH_BASE}"
 ````
 
-Either clone the repo or uncompress latest release.
+Next, you have two options to get the Sighthound services either clone the repo or uncompress latest release:
 
 Option 1: Clone the repo:
 
@@ -32,33 +63,33 @@ RELEASE="v1.3.0"
 git checkout tags/${RELEASE}
 ```
 
-Option 2: Uncompress latest release:
+Option 2: Download and extract the latest release:
 
 ```bash
 RELEASE="v1.3.0"
-mkdir /data/sighthound/services
-cd /data/sighthound/services
+mkdir "${SH_BASE}"/services
+cd "${SH_BASE}"/services
 wget https://github.com/sighthoundinc/services/releases/download/${RELEASE}/sh-services-${RELEASE}.tar.gz
 tar -xvf sh-services-${RELEASE}.tar.gz
 rm sh-services-${RELEASE}.tar.gz
 ```
 
-#### Installing SIO analytics license and Docker registry key
+#### Prerequisit 2 & 3: Installing SIO analytics license and Docker registry key
 
-Copy (cp) or remote secure copy (scp) the Sighthound provided files to the right location:
-```
+Copy (cp) or remote secure copy (scp) the Sighthound provided files to the correct location:
+```bash
 # License
-mkdir -p /data/sighthound/license
-cp ~/Downloads/sighthound-license.json /data/sighthound/license
+mkdir -p "${SH_BASE}"/license
+cp ~/Downloads/sighthound-license.json "${SH_BASE}"/license
 # Docker key
-mkdir -p /data/sighthound/keys
-cp ~/Downloads/sighthound-keyfile.json /data/sighthound/keys
+mkdir -p "${SH_BASE}"/keys
+cp ~/Downloads/sighthound-keyfile.json "${SH_BASE}"/keys
 ```
 
-#### Logging in to the docker registry
+#### Prerequisit 3: Logging into the Docker Registry
 
 ```bash
-docker login -u _json_key -p  "`cat /data/sighthound/keys/sighthound-keyfile.json`" us-central1-docker.pkg.dev
+docker login -u _json_key -p  "`cat "${SH_BASE}"/keys/sighthound-keyfile.json`" us-central1-docker.pkg.dev
 ```
 
 ### Enabling a test RTSP
@@ -69,50 +100,60 @@ If you need to test SIO analytics service and don't have an available RTSP sourc
 ./scripts/sh-services enable live555
 ```
 
-Then copy the test video file to the live555 mount path:
+Next, copy your test video file to the live555 mount path:
 
 ```bash
-mkdir -p ./live555/test-data
+mkdir -p "${SH_BASE}"/services/live555/test-data
 # cp or scp
-cp <my-video>  ./live555/test-data/my-video.mp4
+cp <my-video>  "${SH_BASE}"/services/live555/test-data/my-video.mp4
 ```
 
-And finally, point SIO to live555:
+Finally, enable the live555 SIO configuration:
 
 ```bash
-cp ./sio/examples/live555/* ./sio/conf
-# Check the configuration file, verify paramets and save
-./scripts/sh-services edit sio
+$ ./scripts/sh-services select_example sio live555
 ```
 
-### All devices
+### Running Sighhound services
+
+First check the license:
+
+```bash
+./scripts/sh-services up license
+```
 
-To start, just run:
+To start running the services:
 
 ```bash
 ./scripts/sh-services up all
 ```
 
-and to edit the configuration of services, run:
+To modify the configuration of any service, run the following command and follow the prompts:
 
 ```bash
 ./scripts/sh-services edit all
 ```
 
-### Test
+### Testing the Setup
 
 #### Visual test
 
-You can use SIOOutput example to test that SIO Analytics is working, just run:
+To visually confirm your setup, use the SIOOutput example to create an RTSP feed of your live555 video, annotated with the analytics data.
+
+Just run:
 
 ```bash
 cd ./examples/SIO_RTSP_Output
 docker compose up
 ```
 
-And open VLC at `rtsp://localhost:8554/live`.
+Then, open VLC at rtsp://localhost:8554/live.
 
-## Available services
+This should be sufficient for most users to get started with the Sighthound Services. For more detailed information on individual services, how to change Docker environment variables, and deployment instructions, please see the full documentation below.
+
+## Services in Detail
+
+This section provides more information about each individual service, including their role, exposed ports, and any special instructions for their use.
 
 ### mcp (Media Control Point)
 
@@ -143,12 +184,13 @@ AMQP broker. If the device operates in a standalone mode, must be enabled. If ex
 
 First create the data dirs
 
-1. `mkdir -p /data/sighthound`
-2. `mkdir -p /data/sighthound/media`
-3. `mkdir -p /data/sighthound/services`
-4. `mkdir -p /data/sighthound/license`
-5. Install SIO license in `/data/sighthound/license/sighthound-license.json`
-6. Uncompress services tarball into `/data/sighthound/services`
+0. Set the base path variable `SH_BASE="/data/sighthound"`
+1. `mkdir -p "${SH_BASE}"`
+2. `mkdir -p "${SH_BASE}"/media`
+3. `mkdir -p "${SH_BASE}"/services`
+4. `mkdir -p "${SH_BASE}"/license`
+5. Install SIO license in `"${SH_BASE}"/license/sighthound-license.json`
+6. Uncompress services tarball into `"${SH_BASE}"/services`
 7. Modify the `sio.json` file corresponding your sio selected configuration. (Setting the right URL, pipeline parameters...)
 8. Finally, create the docker .env files by running: `./scripts/sh-services merge all`
 
@@ -176,7 +218,7 @@ For more advanced options visit [VehicleAnalytics Documentation](https://dev.sig
 
 ## Changing Docker env variables
 
-If you need to modify the `.env` file of a service, simply create a new `.env file` like this
+If you need to modify the `.env` file of a service, you can either `./scripts/sh-services edit all` or  create a new `.env file` like this
 
 ```bash
 echo "SIO_DOCKER_TAG=r221202" >  sio/conf/0009-debug.env
@@ -185,14 +227,13 @@ echo "SIO_DOCKER_TAG=r221202" >  sio/conf/0009-debug.env
 and then update the services:
 
 ```bash
-bash ./scripts/sh-services merge all
+./scripts/sh-services merge all
 ```
 
 ## Deployment
 
 ```bash
-docker network create sh-device-ui_sh-ui-net || true
-bash ./scripts/sh-services up all
+./scripts/sh-services up all
 ```
 
 At this point you can test your deployment by going to:
@@ -205,7 +246,7 @@ See development example and demonstration scripts at [docs/examples](docs/exampl
 
 ## Tips and tricks
 
-For using `sh-services` you may want to run: `export PATH=${PATH}:/data/sighthound/services/scripts` first.
+For using `sh-services` you may want to run: `export PATH=${PATH}:"${SH_BASE}"/services/scripts` first.
 
 Then you can do commands like:
 
@@ -218,7 +259,7 @@ sh-services up all
 To disable a service just run:
 
 ```bash
-sh-services disable <service>
+./sh-services disable <service>
 # OR
 touch <service>/disabled
 ```
@@ -228,7 +269,9 @@ touch <service>/disabled
 To enalbe a service just run:
 
 ```bash
-sh-services enable <service>
+./sh-services enable <service>
 # OR
 rm <service>/disabled
-```
+```
+
+This guide should help you get started with Sighthound Services and allow you to efficiently use, manage, and deploy these services. If any part of the guide needs clarification, or if you encounter any issues while using the services, please let us know so we can improve the documentation and address the problem.

VERSION

@@ -1 +1 @@
-v1.3.2
+v1.4.0

amqp-stats/README.md

@@ -0,0 +1,2 @@
+# AMQP Stats
+short description: amqp stats is an example simple SIO pipeline viewer.

amqp-stats/conf/default.env

@@ -2,6 +2,4 @@ AMQP_STATS_DOCKER_IMAGE=us-central1-docker.pkg.dev/ext-edge-analytics/docker/amq
 AMQP_STATS_DOCKER_TAG=1.2.3
 AMQP_STATS_CONTAINER_NAME=amqp-stats
 SERVICE_GROUP=-dev
-MEDIA_PATH=/data/sighthound/media
-LOG_PATH=/data/sighthound/logs/mcp
 CONTENT=message,stats

amqp-stats/docker-compose.yml

@@ -7,7 +7,7 @@ services:
     restart: unless-stopped
     volumes:
       - ${MEDIA_PATH}:/data/sighthound/media:rw
-      - ${LOG_PATH}:/data/sighthound/logs/mcp:rw
+      - ${LOGS_PATH}/amqp-stats:/data/sighthound/logs/mcp:rw
     env_file:
       - .env
     networks:

examples/AqueductRunner/Dockerfile

@@ -0,0 +1,13 @@
+FROM python:3.8
+
+RUN apt update && apt install -y curl
+RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && /usr/bin/python3 get-pip.py
+
+WORKDIR /usr/src/app
+COPY requirements.txt /usr/src/app/
+RUN /usr/bin/python3 -m pip install -r requirements.txt
+
+ENV PYTHONPATH=/usr/src/app/:/usr/src/app/lib/
+ENV PYTHONUNBUFFERED=1
+
+ENTRYPOINT [ "/usr/bin/python3", "/usr/src/app/aqueductRunner.py"]