Skip to content

Commit

Permalink
Add wrapper deployment example and documentation.
Browse files Browse the repository at this point in the history
  • Loading branch information
matthewwardrop committed Nov 30, 2018
1 parent d1a17a9 commit 50f3c74
Show file tree
Hide file tree
Showing 8 changed files with 147 additions and 1 deletion.
17 changes: 16 additions & 1 deletion docs/deployment.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,19 @@
Deployment
==========

Coming soon.
While Omniduct can be used on its own by manually constructing the services that
you need as part of your scripts and packages, it was designed specifically to
integrate well into a organisation-specific Python wrapper package that
preconfigures the services available within that organisation environment.
Typically such deployments would take advantage of Omniduct's `DuctRegistry` to
conveniently expose services within such a package.

An example wrapper package `is provided alongside the omniduct module`__ to help
bootstrap your own wrappers.

.. __: https://github.com/airbnb/omniduct/tree/master/example_wrapper

If you need any assistance, please do not hesitate to reach out to us via the
`GitHub issue tracker`__.

.. __: https://github.com/airbnb/omniduct/issues
1 change: 1 addition & 0 deletions example_wrapper/MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include example_wrapper/services.yml
20 changes: 20 additions & 0 deletions example_wrapper/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Example Omniduct Wrapper

This is an example wrapper package around Omniduct to pre-configure services
availabe within a given organisation. Creating an organisation-specific Omniduct
configuration is a convenient way for people to make use of your organisation's
services via Python.

Services can be accessed using:
```
example_wrapper.services.databases.presto
```

Some services (in this example, `presto`) can been promoted to the top-level
module, and then can also be accessed using:
```
example_wrapper.presto
```

If you need any guidance when building your own wrapper, please open an issue in
the [GitHub issue tracker](https://github.com/airbnb/omniduct/issues).
34 changes: 34 additions & 0 deletions example_wrapper/example_wrapper/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
from collections import OrderedDict

from omniduct.utils.about import show_about as _show_about

from ._version import __author__, __author_email__, __version__ # noqa: F401
from .services import config, logger, registry # noqa: F401


# Expose services via a proxy object
services = registry.get_proxy(by_kind=True)

# For convenience, add some services to the top-level module
registry.populate_namespace(
namespace=globals(),
names=['presto']
)


def about():
"""
Show information about this package.
"""
_show_about(
name='Example Wrapper',
version=__version__,
maintainers=OrderedDict(zip(
[a.strip() for a in __author__.split(',')],
[a.strip() for a in __author_email__.split(',')]
)),
description="""
A simple example wrapper around Omniduct for pre-configuring services
in the context of an organisation.
"""
)
9 changes: 9 additions & 0 deletions example_wrapper/example_wrapper/_version.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
__version__ = '1.0.0'
__author__ = 'John Smith'
__author_email__ = '[email protected]'

__dependencies__ = [
# It is recommended to pin omniduct to a specific major version, and manually repin when updates go out
# Omniduct is quite stable, but this provides a stronger guarantee of stability for users of your package
'omniduct[ssh,presto,s3]>=1.0.0<1.1.0'
]
16 changes: 16 additions & 0 deletions example_wrapper/example_wrapper/services.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import os
import yaml

from omniduct import config, logger, DuctRegistry


SERVICES_CONFIG_PATH = os.path.join(os.path.dirname(__file__), 'services.yml')

# Set up Omniduct configuration path
config._config_path = '~/.example_wrapper/config'

# Build registry from configuration
# Note: If you need to transform the configuration before importing, you can
# directly load the configuration into a dictionary using the `yaml` package,
# modify it before passing it in as the configuration below.
registry = DuctRegistry(config=SERVICES_CONFIG_PATH)
27 changes: 27 additions & 0 deletions example_wrapper/example_wrapper/services.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
caches:
cache: # Setup a local cache for use by services
protocol: filesystem_cache
fs: local
path: ~/.example_wrapper/cache

databases:
presto:
protocol: presto
host: presto.company.com
cache: cache
remote: gateway

filesystems:
local:
protocol: localfs
global_writes: True
s3_company_bucket:
protocol: s3
bucket: company-bucket

remotes:
gateway: # A remote gateway used to access Presto services
protocol: ssh
host: gateway.company.com
smartcards:
yubikey: /usr/local/lib/opensc-pkcs11.so
24 changes: 24 additions & 0 deletions example_wrapper/setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
from setuptools import find_packages, setup

# Extract version information from Omniduct _version.py
version_info = {}
with open('example_wrapper/_version.py') as version_file:
exec(version_file.read(), version_info)

setup(
# Package metadata
name="example_wrapper",
version=version_info['__version__'],
author=version_info['__author__'],
author_email=version_info['__author_email__'],
url='http://git.company.com/example_wrapper',
description="Exposes Company.com services locally.",
license='Internal',

# Package details
packages=find_packages(),
include_package_data=True,

# Dependencies
install_requires=version_info['__dependencies__']
)

0 comments on commit 50f3c74

Please sign in to comment.