README

*****Preliminaries
// Remote debug config...

1. export VERTX_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005"
2. vertx runMod modulename -conf jsonconf -ha -hagroup optional 
3. run remote debug in IDE (intelij is preferred multiple tabs concurrently)

Geia sas to kainourio project gia to query processor einai etoimo opws epishs 
kai ta plugins.

To repo einai sto github kai einai private pros to paron to link einai 
https://github.com/vagvaz/leads-query-processor.git

Gia version control tha xrhsimopoihsoume to git. Akolouthoun merika tutorials 
to prwto einai syntomo kai periektiko to 2o gia to github kai to 3o genikes gnwseis xrhsimes!!!
git tutorial
http://www.vogella.com/tutorials/Git/article.html#gitdefintion_localrepositories
github tutorial
https://help.github.com/articles/set-up-git		      
general knowledge
https://www.atlassian.com/git/tutorial/git-basics

o kathe enas sas tha exei 1 branch
vagvaz -> vagvaz
lefteris -> lefteris
ioakeim   -> ioakeim
giannis    -> giannis
odysseas   -> odysseas
npavlakis   -> npavlakis

thelw na mou steilete ta username sas sto github gia na sas kanw add. 
Tha douleyete sto proswpiko sas branch kai otan teleiwnete kapoio feature tha 
kanete pull request

pull request https://help.github.com/articles/creating-a-pull-request

an thelete mporeite na kanete locally osa branch thelete. Tha protimousa oti 
allages exete na tis pernate sto proswpika sas  kai na kanete pull request wste 
na enswmatwnw tis allages sas.

*** gia na kanete push to branch sas 
git push -u origin branchname

Epishs sas parakalw issues provlhmata bugs pou den diorthwnete mhn mou ta lete 
opws pernate apo to grafeio
dhmiourghste new issue sto repository.

*****Project

afou kanete clone to project tha deite oti uparxoun diaforoi fakeloi praktika 
ena gia kathe leitourgikh monada:
common: koina xrhsima pragmata
core:   ta apolutws aparaithta
crawler: o crawler pou eftiaksa gia na testarete ta plugins kai genika
deployer: o deployer
nqe:  node query executor
parent: voithitiko project gia na mporoume na kanoume build
planner: o planner
plugin-examples: aplws periexei to template maven project gia ta plugins kai to 
sentiment and transform 			plugin gia reference
plugins: kwdikas related sta plugins

* Compile
gia na kanete compile to project mvn compile
* create jar. prosekste auto to jar einai MONO o kwdikas kai ta resources oxi 
kai ta dependencies
mvn package
* create a super jar, dhmiourgeitai ena jar -with-dependencies me auto mporeite 
na treksete to kwdika sas 
mvn assembly:single

* Developping
Exw valei sxolia genika stis klasseis pou tha xreiasteite mind

---- in commmon -----
LQPConfiguration (kaleita panta thn initialize wste na exete working cluster 
ready infinispan)
Tuple  tuple utility class for handling json objects
StringConstants
classes in utils folder
InfinispanManager interface that abstracts Infinispan operators getCache bla bla
InfinispanClusterSingleton.java xrhsimopoihste authn thn klash gia na apokthsete 
prosvash se 1 InfinispanManager



--- in plugins ----
PluginBaseImpl ena basic plugin pou kanei log oti mpainei kai oti vgainei apo 
mia getCache
PluginInterface to interface pou tha prepei na ulopoioun ta plugins
PluginManager h klash pou xrhsimopoieite gia na kanete upload kai deploy plugins

--- plugin-examples ---
plugins-template folder pou xrhsimopoieite san base gia na kanete to plugin sas 
(NA ALLAZETE to artifactId)
		  mesa epishs uparxei kai enas fakelos test me 2 klaseis mia gia 
testing gia local kai gia clustered. 
sentiment plugin  to gnwsto sentiment and entity extraction plugin kanei compile 
alla tou leipoun oi classifiers 77Mb ta exw anevasei edw http://goo.gl/4dfs8j 
ama thelete na to treksete

transform-plugin aplo plugin pou aplws filtrarei kapoia attributes


---- Pws uloipoieitai ena plugin
1. sto leads-query-processor project: mvn clean install 
   etsi kanete compile kai egkathistate sto local maven repository sas ola ta 
dependencies tou project.
2. copy to fakelo plugin-examples/plugins-template se ena kainourio fakelo px 
myplugin
3. customize myplugin/pom.xml (ALLAZETE to artifactId, kai prostetete ta maven 
deps)
4. start happy coding....

ta plugins ta exw gia reference mporeite na deite pws exw ulopoihsei auta ta 3 
plugins. gia test exw kanei modify ton crawler pou eixe ftiaksei o Pierre wste 
na gemizoume mia cache deite pws ton xrhsimmpoiw gia kapoio logo  to 
crawler.seed configuration den douleyei 

---- PROSOXH
1. epeidh evala na uparxoun file stores sta caches (shmainei oti swzontai se 
arxeio) sto /tmp/ tha vriskontai arxeia cacheName.dat ta svhnete...
2. mhn ksegelasteite kanontas iterations panw apo ena cache me to entrySet. Sto 
deployment version den tha exete ola ta data gia na douleyei swsta...(twra 
douleyei giati ola ta instances koitazoun to idio arxeio...)
3. to state sas an to allaksete prepei na to ksanavalete sto KVS

4. gia na testarete to clustered deployment prepei prwta na ftiaksete 1 jar me 
ola ta dependencies mvn assembly:single) kai ena xml configuration arxeio. auta


--UPLOADING (new)
-Setup
install swift, sudo apt-get install python-swiftclient
get the LEADS-openrc.sh from the C&H Dashboard->Access & Security->API Access
run it and set the password 
get the repo from https://github.com/skarab7/leads_query-engine.git (check the branches I used develop)
make upload.sh executable


-Execution
run "source pathto/tools/openstack_cli/bin/activate"
run ./upload.sh

-Extended Setup (if above does not work)
check the URL of the cluster tha you are uploading hamm5,6 !
run "source pathto/tools/openstack_cli/bin/activate"
pip install -r  openstack_cli/requirements.txt
install necessary libraries if above fails
eg. sudo apt-get install libffi-dev libssl-dev python-dev
good luck