Wednesday, November 9, 2016

Apache Apex example

To create example apex app using maven:
step 1:
mvn -B archetype:generate -DarchetypeGroupId=org.apache.apex -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion="3.3.0-incubating"  -DgroupId=com.example  -Dpackage=com.example.myapexapp  -DartifactId=myapexapp  -Dversion=1.0-SNAPSHOT

step2:
download https://github.com/srccodes/hadoop-common-2.2.0-bin/archive/master.zip and set hadoop.home.dir var in ApplicationTest before running test class
ApplicationTest.java
------------------------------------
Terminologies in apex:
Operators: which actually do data processing like kpi  calculation. can have many Operators in data processing
Unifier: which will do merge of partial aggregation data make final aggregation results (sum,count..etc). can have many Unifier in data processing
checkpoint: to store intermediate result, which will used in fault-tolerance time. default is 30 sec

No comments: