-
Notifications
You must be signed in to change notification settings - Fork 46
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add wrapper deployment example and documentation.
- Loading branch information
1 parent
d1a17a9
commit 50f3c74
Showing
8 changed files
with
147 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,19 @@ | ||
Deployment | ||
========== | ||
|
||
Coming soon. | ||
While Omniduct can be used on its own by manually constructing the services that | ||
you need as part of your scripts and packages, it was designed specifically to | ||
integrate well into a organisation-specific Python wrapper package that | ||
preconfigures the services available within that organisation environment. | ||
Typically such deployments would take advantage of Omniduct's `DuctRegistry` to | ||
conveniently expose services within such a package. | ||
|
||
An example wrapper package `is provided alongside the omniduct module`__ to help | ||
bootstrap your own wrappers. | ||
|
||
.. __: https://github.com/airbnb/omniduct/tree/master/example_wrapper | ||
|
||
If you need any assistance, please do not hesitate to reach out to us via the | ||
`GitHub issue tracker`__. | ||
|
||
.. __: https://github.com/airbnb/omniduct/issues |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
include example_wrapper/services.yml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Example Omniduct Wrapper | ||
|
||
This is an example wrapper package around Omniduct to pre-configure services | ||
availabe within a given organisation. Creating an organisation-specific Omniduct | ||
configuration is a convenient way for people to make use of your organisation's | ||
services via Python. | ||
|
||
Services can be accessed using: | ||
``` | ||
example_wrapper.services.databases.presto | ||
``` | ||
|
||
Some services (in this example, `presto`) can been promoted to the top-level | ||
module, and then can also be accessed using: | ||
``` | ||
example_wrapper.presto | ||
``` | ||
|
||
If you need any guidance when building your own wrapper, please open an issue in | ||
the [GitHub issue tracker](https://github.com/airbnb/omniduct/issues). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
from collections import OrderedDict | ||
|
||
from omniduct.utils.about import show_about as _show_about | ||
|
||
from ._version import __author__, __author_email__, __version__ # noqa: F401 | ||
from .services import config, logger, registry # noqa: F401 | ||
|
||
|
||
# Expose services via a proxy object | ||
services = registry.get_proxy(by_kind=True) | ||
|
||
# For convenience, add some services to the top-level module | ||
registry.populate_namespace( | ||
namespace=globals(), | ||
names=['presto'] | ||
) | ||
|
||
|
||
def about(): | ||
""" | ||
Show information about this package. | ||
""" | ||
_show_about( | ||
name='Example Wrapper', | ||
version=__version__, | ||
maintainers=OrderedDict(zip( | ||
[a.strip() for a in __author__.split(',')], | ||
[a.strip() for a in __author_email__.split(',')] | ||
)), | ||
description=""" | ||
A simple example wrapper around Omniduct for pre-configuring services | ||
in the context of an organisation. | ||
""" | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
__version__ = '1.0.0' | ||
__author__ = 'John Smith' | ||
__author_email__ = '[email protected]' | ||
|
||
__dependencies__ = [ | ||
# It is recommended to pin omniduct to a specific major version, and manually repin when updates go out | ||
# Omniduct is quite stable, but this provides a stronger guarantee of stability for users of your package | ||
'omniduct[ssh,presto,s3]>=1.0.0<1.1.0' | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
import os | ||
import yaml | ||
|
||
from omniduct import config, logger, DuctRegistry | ||
|
||
|
||
SERVICES_CONFIG_PATH = os.path.join(os.path.dirname(__file__), 'services.yml') | ||
|
||
# Set up Omniduct configuration path | ||
config._config_path = '~/.example_wrapper/config' | ||
|
||
# Build registry from configuration | ||
# Note: If you need to transform the configuration before importing, you can | ||
# directly load the configuration into a dictionary using the `yaml` package, | ||
# modify it before passing it in as the configuration below. | ||
registry = DuctRegistry(config=SERVICES_CONFIG_PATH) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
caches: | ||
cache: # Setup a local cache for use by services | ||
protocol: filesystem_cache | ||
fs: local | ||
path: ~/.example_wrapper/cache | ||
|
||
databases: | ||
presto: | ||
protocol: presto | ||
host: presto.company.com | ||
cache: cache | ||
remote: gateway | ||
|
||
filesystems: | ||
local: | ||
protocol: localfs | ||
global_writes: True | ||
s3_company_bucket: | ||
protocol: s3 | ||
bucket: company-bucket | ||
|
||
remotes: | ||
gateway: # A remote gateway used to access Presto services | ||
protocol: ssh | ||
host: gateway.company.com | ||
smartcards: | ||
yubikey: /usr/local/lib/opensc-pkcs11.so |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
from setuptools import find_packages, setup | ||
|
||
# Extract version information from Omniduct _version.py | ||
version_info = {} | ||
with open('example_wrapper/_version.py') as version_file: | ||
exec(version_file.read(), version_info) | ||
|
||
setup( | ||
# Package metadata | ||
name="example_wrapper", | ||
version=version_info['__version__'], | ||
author=version_info['__author__'], | ||
author_email=version_info['__author_email__'], | ||
url='http://git.company.com/example_wrapper', | ||
description="Exposes Company.com services locally.", | ||
license='Internal', | ||
|
||
# Package details | ||
packages=find_packages(), | ||
include_package_data=True, | ||
|
||
# Dependencies | ||
install_requires=version_info['__dependencies__'] | ||
) |