Skip to content

nmadhire/metacat

This branch is 251 commits behind Netflix/metacat:master.

Folders and files

NameName
Last commit message
Last commit date
Feb 20, 2018
Jan 12, 2018
May 17, 2017
Sep 5, 2018
Nov 14, 2018
Oct 17, 2018
Oct 12, 2018
Oct 12, 2018
Nov 16, 2018
Oct 12, 2018
Mar 16, 2018
Nov 9, 2018
Mar 7, 2018
Mar 7, 2018
Oct 12, 2018
May 17, 2018
Oct 30, 2018
Nov 5, 2018
Oct 10, 2018
Nov 16, 2018
Oct 12, 2018
Jun 21, 2017
Jun 6, 2018
Dec 5, 2016
May 11, 2017
Jun 28, 2017
Mar 19, 2016
Mar 19, 2016
Aug 3, 2017
Nov 14, 2018
May 21, 2018
Jun 6, 2018
Jun 19, 2017
May 11, 2017
May 11, 2017
Mar 23, 2016
Jan 30, 2018

Repository files navigation

Metacat

Download License Issues NetflixOSS Lifecycle

Introduction

Metacat is a unified metadata exploration API service. You can explore Hive, RDS, Teradata, Redshift, S3 and Cassandra. Metacat provides you information about what data you have, where it resides and how to process it. Metadata in the end is really data about the data. So the primary purpose of Metacat is to give a place to describe the data so that we could do more useful things with it.

Metacat focusses on solving these three problems:

  • Federate views of metadata systems.
  • Allow arbitrary metadata storage about data sets.
  • Metadata discovery

Documentation

TODO

Releases

Releases

Builds

Metacat builds are run on Travis CI here. Build Status

Getting Started

git clone git@github.com:Netflix/metacat.git
cd metacat
./gradlew clean build

Once the build is completed, the metacat WAR file is generated under metacat-war/build/libs directory. Metacat needs two basic configurations:

  • metacat.plugin.config.location: Path to the directory containing the catalog configuration. Please look at catalog samples used for functional testing.
  • metacat.usermetadata.config.location: Path to the configuration file containing the connection properties to store user metadata. Please look at this sample.

Running Locally

Take the build WAR in metacat-war/build/libs and deploy it to an existing Tomcat as ROOT.war.

The REST API can be accessed @ http://localhost:8080/mds/v1/catalog

Swagger API documentation can be accessed @ http://localhost:8080/swagger-ui.html

Docker Compose Example

Pre-requisite: Docker compose is installed

To start a self contained Metacat environment with some sample catalogs run the command below. This will start a docker-compose cluster containing a Metacat container, a Hive Metastore Container, a Cassandra container and a PostgreSQL container.

./gradlew metacatPorts
  • metacatPorts - Prints out what exposed ports are mapped to the internal container ports. Look for the mapped port (MAPPED_PORT) to port 8080.

REST API can be accessed @ http://localhost:<MAPPED_PORT>/mds/v1/catalog

Swagger API documentation can be accessed @ http://localhost:<MAPPED_PORT>/swagger-ui.html

To stop the docker compose cluster:

./gradlew stopMetacatCluster

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Java 71.7%
  • Groovy 27.4%
  • Other 0.9%