07:11:45,659 INFO Test worker context.TestContextManager - @TestExecutionListeners is not present for class [class org.springframework.data.hadoop.pig.PigBatchTest]: using defaults.
07:11:45,661 INFO Test worker xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/pig/batch.xml]
07:11:45,689 INFO Test worker xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/batch-common.xml]
07:11:45,702 INFO Test worker xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/hadoop-ctx.xml]
07:11:45,721 INFO Test worker support.DefaultListableBeanFactory - Overriding bean definition for bean 'mainJob': replacing [Generic bean: class [org.springframework.batch.core.configuration.xml.SimpleFlowFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null] with [Generic bean: class [org.springframework.batch.core.configuration.xml.JobParserJobFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null]
07:11:45,726 INFO Test worker support.GenericApplicationContext - Refreshing org.springframework.context.support.GenericApplicationContext@104fb34: startup date [Thu Sep 05 07:11:45 PDT 2013]; root of context hierarchy
07:11:45,743 INFO Test worker config.PropertyPlaceholderConfigurer - Loading properties file from class path resource [test.properties]
07:11:45,750 INFO Test worker support.DefaultListableBeanFactory - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@1354884c: defining beans [jobRepository,transactionManager,jobLauncher,taskExecutor,ppc,hadoopFs,hadoopResourceLoader,hadoopConfiguration,cfg-init,fs-init,rl-init,org.springframework.data.hadoop.scripting.HdfsScriptRunner#0,org.springframework.data.hadoop.scripting.HdfsScriptRunner#1,org.springframework.batch.core.scope.internalStepScope,org.springframework.beans.factory.config.CustomEditorConfigurer,org.springframework.batch.core.configuration.xml.CoreNamespacePostProcessor,bean,do-pig,mainJob,tasklet,pigFactory,pig-script,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor]; root of factory hierarchy
07:11:45,905 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:45,919 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:45,929 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:45,944 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:46,631 INFO Test worker pigstats.ScriptState - Pig features used in the script: UNKNOWN
07:11:46,662 INFO Test worker rules.ColumnPruneVisitor - Columns pruned for A: $1
07:11:46,745 INFO Test worker mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
07:11:46,767 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
07:11:46,768 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
07:11:46,782 INFO Test worker pigstats.ScriptState - Pig script settings are added to the job
07:11:46,787 INFO Test worker mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
07:11:46,787 INFO Test worker mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
07:11:46,788 INFO Test worker mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=3045
07:11:46,788 INFO Test worker mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
07:11:46,798 INFO Test worker mapReduceLayer.JobControlCompiler - Setting up single store job
07:11:46,802 INFO Test worker data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
07:11:46,802 INFO Test worker data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
07:11:46,802 INFO Test worker data.SchemaTupleFrontend - Distributed cache not supported or needed in local mode. Setting key [pig.schematuple.local.dir] with code temp directory: /tmp/1378390306802-0
07:11:46,824 INFO Test worker mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
07:11:46,826 WARN JobControl mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
07:11:46,827 WARN JobControl mapred.JobClient - No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
07:11:46,847 INFO JobControl input.FileInputFormat - Total input paths to process : 1
07:11:46,847 INFO JobControl util.MapRedUtil - Total input paths to process : 1
07:11:46,848 INFO JobControl util.MapRedUtil - Total input paths (combined) to process : 1
07:11:46,887 INFO Thread-582 mapred.LocalJobRunner - Waiting for map tasks
07:11:46,887 INFO pool-2-thread-1 mapred.LocalJobRunner - Starting task: attempt_local1012089835_0002_m_000000_0
07:11:46,929 INFO pool-2-thread-1 util.ProcessTree - setsid exited with exit code 0
07:11:46,943 INFO pool-2-thread-1 mapred.Task - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@59a6acb9
07:11:46,947 INFO pool-2-thread-1 mapred.MapTask - Processing split: Number of splits :1
Total Length = 3045
Input split[0]:
Length = 3045
Locations:
-----------------------
07:11:46,951 INFO pool-2-thread-1 mapReduceLayer.PigRecordReader - Current split being processed file:/data/bamboo-home/xml-data/build-dir/SPRINGDATAHADOOP-HDP13CENTOS0-JOB1/src/test/resources/logs/apache_access.log:0+3045
07:11:46,963 INFO pool-2-thread-1 data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
07:11:46,968 INFO pool-2-thread-1 mapReduceLayer.PigMapOnly$Map - Aliases being processed per job phase (AliasName[line,offset]): M: A[1,4],B[-1,-1] C: R:
07:11:46,974 INFO pool-2-thread-1 mapred.Task - Task:attempt_local1012089835_0002_m_000000_0 is done. And is in the process of commiting
07:11:46,978 INFO pool-2-thread-1 mapred.LocalJobRunner -
07:11:46,978 INFO pool-2-thread-1 mapred.Task - Task attempt_local1012089835_0002_m_000000_0 is allowed to commit now
07:11:46,980 INFO pool-2-thread-1 output.FileOutputCommitter - Saved output of task 'attempt_local1012089835_0002_m_000000_0' to file:/tmp/temp323995783/tmp470645224
07:11:46,981 INFO pool-2-thread-1 mapred.LocalJobRunner -
07:11:46,981 INFO pool-2-thread-1 mapred.Task - Task 'attempt_local1012089835_0002_m_000000_0' done.
07:11:46,981 INFO pool-2-thread-1 mapred.LocalJobRunner - Finishing task: attempt_local1012089835_0002_m_000000_0
07:11:46,981 INFO Thread-582 mapred.LocalJobRunner - Map task executor complete.
07:11:47,325 INFO Test worker mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local1012089835_0002
07:11:47,326 INFO Test worker mapReduceLayer.MapReduceLauncher - Processing aliases A,B
07:11:47,326 INFO Test worker mapReduceLayer.MapReduceLauncher - detailed locations: M: A[1,4],B[-1,-1] C: R:
07:11:47,334 WARN Test worker pigstats.PigStatsUtil - Failed to get RunningJob for job job_local1012089835_0002
07:11:47,339 INFO Test worker mapReduceLayer.MapReduceLauncher - 100% complete
07:11:47,339 INFO Test worker pigstats.SimplePigStats - Detected Local mode. Stats reported below may be incomplete
07:11:47,341 INFO Test worker pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
1.2.0 0.11.0 bamboo 2013-09-05 07:11:46 2013-09-05 07:11:47 UNKNOWN
Success!
Job Stats (time in seconds):
JobId Alias Feature Outputs
job_local1012089835_0002 A,B MAP_ONLY file:/tmp/temp323995783/tmp470645224,
Input(s):
Successfully read records from: "file:///data/bamboo-home/xml-data/build-dir/SPRINGDATAHADOOP-HDP13CENTOS0-JOB1/src/test/resources/logs/apache_access.log"
Output(s):
Successfully stored records in: "file:/tmp/temp323995783/tmp470645224"
Job DAG:
job_local1012089835_0002
07:11:47,343 INFO Test worker mapReduceLayer.MapReduceLauncher - Success!
07:11:47,348 WARN Test worker data.SchemaTupleBackend - SchemaTupleBackend has already been initialized
07:11:47,352 INFO Test worker input.FileInputFormat - Total input paths to process : 1
07:11:47,353 INFO Test worker util.MapRedUtil - Total input paths to process : 1
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET / HTTP/1.1" 500 606 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:53:10 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/1.0" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /style.css HTTP/1.1" 200 614 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /img/pti-round.jpg HTTP/1.1" 200 17524 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:21 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET / HTTP/1.1" 200 34 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET /favicon.ico HTTP/1.1" 200 11514 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:53 +0300] "GET /cgi/pti.pl HTTP/1.1" 500 617 "http:/contact.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/0.9" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:27 +0300] "GET / HTTP/1.1" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:34 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:45 +0300] "GET /talks/Fundamentals/read-excel-file.html HTTP/1.1" 404 311 "http://pti.local/unix_sysadmin.html" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
07:11:47,392 INFO Test worker support.SimpleJobLauncher - Job: [FlowJob: [name=mainJob]] launched with the following parameters: [{}]
07:11:47,408 INFO Test worker job.SimpleStepHandler - Executing step: [bean]
07:11:47,424 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:47,434 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:47,444 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:47,454 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:47,501 INFO Test worker pigstats.ScriptState - Pig features used in the script: UNKNOWN
07:11:47,503 INFO Test worker rules.ColumnPruneVisitor - Columns pruned for A: $1
07:11:47,507 INFO Test worker mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
07:11:47,509 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
07:11:47,509 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
07:11:47,511 INFO Test worker pigstats.ScriptState - Pig script settings are added to the job
07:11:47,511 INFO Test worker mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
07:11:47,512 INFO Test worker mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
07:11:47,514 INFO Test worker mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=3045
07:11:47,514 INFO Test worker mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
07:11:47,522 INFO Test worker mapReduceLayer.JobControlCompiler - Setting up single store job
07:11:47,523 INFO Test worker data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
07:11:47,523 INFO Test worker data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
07:11:47,524 INFO Test worker data.SchemaTupleFrontend - Distributed cache not supported or needed in local mode. Setting key [pig.schematuple.local.dir] with code temp directory: /tmp/1378390307523-0
07:11:47,538 INFO Test worker mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
07:11:47,542 WARN JobControl mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
07:11:47,544 WARN JobControl mapred.JobClient - No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
07:11:47,564 INFO JobControl input.FileInputFormat - Total input paths to process : 1
07:11:47,564 INFO JobControl util.MapRedUtil - Total input paths to process : 1
07:11:47,565 INFO JobControl util.MapRedUtil - Total input paths (combined) to process : 1
07:11:47,639 INFO Thread-588 mapred.LocalJobRunner - Waiting for map tasks
07:11:47,639 INFO pool-3-thread-1 mapred.LocalJobRunner - Starting task: attempt_local569144188_0003_m_000000_0
07:11:47,656 INFO pool-3-thread-1 mapred.Task - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@56300388
07:11:47,658 INFO pool-3-thread-1 mapred.MapTask - Processing split: Number of splits :1
Total Length = 3045
Input split[0]:
Length = 3045
Locations:
-----------------------
07:11:47,662 INFO pool-3-thread-1 mapReduceLayer.PigRecordReader - Current split being processed file:/data/bamboo-home/xml-data/build-dir/SPRINGDATAHADOOP-HDP13CENTOS0-JOB1/src/test/resources/logs/apache_access.log:0+3045
07:11:47,673 INFO pool-3-thread-1 data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
07:11:47,679 INFO pool-3-thread-1 mapReduceLayer.PigMapOnly$Map - Aliases being processed per job phase (AliasName[line,offset]): M: A[1,4],B[-1,-1] C: R:
07:11:47,683 INFO pool-3-thread-1 mapred.Task - Task:attempt_local569144188_0003_m_000000_0 is done. And is in the process of commiting
07:11:47,685 INFO pool-3-thread-1 mapred.LocalJobRunner -
07:11:47,685 INFO pool-3-thread-1 mapred.Task - Task attempt_local569144188_0003_m_000000_0 is allowed to commit now
07:11:47,688 INFO pool-3-thread-1 output.FileOutputCommitter - Saved output of task 'attempt_local569144188_0003_m_000000_0' to file:/tmp/temp2079259380/tmp-1132905537
07:11:47,689 INFO pool-3-thread-1 mapred.LocalJobRunner -
07:11:47,689 INFO pool-3-thread-1 mapred.Task - Task 'attempt_local569144188_0003_m_000000_0' done.
07:11:47,690 INFO pool-3-thread-1 mapred.LocalJobRunner - Finishing task: attempt_local569144188_0003_m_000000_0
07:11:47,690 INFO Thread-588 mapred.LocalJobRunner - Map task executor complete.
07:11:48,039 INFO Test worker mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local569144188_0003
07:11:48,039 INFO Test worker mapReduceLayer.MapReduceLauncher - Processing aliases A,B
07:11:48,039 INFO Test worker mapReduceLayer.MapReduceLauncher - detailed locations: M: A[1,4],B[-1,-1] C: R:
07:11:48,047 WARN Test worker pigstats.PigStatsUtil - Failed to get RunningJob for job job_local569144188_0003
07:11:48,047 INFO Test worker mapReduceLayer.MapReduceLauncher - 100% complete
07:11:48,047 INFO Test worker pigstats.SimplePigStats - Detected Local mode. Stats reported below may be incomplete
07:11:48,048 INFO Test worker pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
1.2.0 0.11.0 bamboo 2013-09-05 07:11:47 2013-09-05 07:11:48 UNKNOWN
Success!
Job Stats (time in seconds):
JobId Alias Feature Outputs
job_local569144188_0003 A,B MAP_ONLY file:/tmp/temp2079259380/tmp-1132905537,
Input(s):
Successfully read records from: "file:///data/bamboo-home/xml-data/build-dir/SPRINGDATAHADOOP-HDP13CENTOS0-JOB1/src/test/resources/logs/apache_access.log"
Output(s):
Successfully stored records in: "file:/tmp/temp2079259380/tmp-1132905537"
Job DAG:
job_local569144188_0003
07:11:48,049 INFO Test worker mapReduceLayer.MapReduceLauncher - Success!
07:11:48,050 WARN Test worker data.SchemaTupleBackend - SchemaTupleBackend has already been initialized
07:11:48,052 INFO Test worker input.FileInputFormat - Total input paths to process : 1
07:11:48,052 INFO Test worker util.MapRedUtil - Total input paths to process : 1
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET / HTTP/1.1" 500 606 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:53:10 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/1.0" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /style.css HTTP/1.1" 200 614 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /img/pti-round.jpg HTTP/1.1" 200 17524 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:21 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET / HTTP/1.1" 200 34 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET /favicon.ico HTTP/1.1" 200 11514 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:53 +0300] "GET /cgi/pti.pl HTTP/1.1" 500 617 "http:/contact.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/0.9" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:27 +0300] "GET / HTTP/1.1" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:34 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:45 +0300] "GET /talks/Fundamentals/read-excel-file.html HTTP/1.1" 404 311 "http://pti.local/unix_sysadmin.html" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
07:11:48,100 INFO Test worker job.SimpleStepHandler - Executing step: [do-pig]
07:11:48,118 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:48,129 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:48,139 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:48,149 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
07:11:48,194 INFO Test worker pigstats.ScriptState - Pig features used in the script: UNKNOWN
07:11:48,196 INFO Test worker rules.ColumnPruneVisitor - Columns pruned for A: $1
07:11:48,200 INFO Test worker mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
07:11:48,201 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
07:11:48,202 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
07:11:48,204 INFO Test worker pigstats.ScriptState - Pig script settings are added to the job
07:11:48,205 INFO Test worker mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
07:11:48,205 INFO Test worker mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
07:11:48,206 INFO Test worker mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=3045
07:11:48,206 INFO Test worker mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
07:11:48,213 INFO Test worker mapReduceLayer.JobControlCompiler - Setting up single store job
07:11:48,214 INFO Test worker data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
07:11:48,214 INFO Test worker data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
07:11:48,214 INFO Test worker data.SchemaTupleFrontend - Distributed cache not supported or needed in local mode. Setting key [pig.schematuple.local.dir] with code temp directory: /tmp/1378390308214-0
07:11:48,222 INFO Test worker mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
07:11:48,225 WARN JobControl mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
07:11:48,226 WARN JobControl mapred.JobClient - No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
07:11:48,240 INFO JobControl input.FileInputFormat - Total input paths to process : 1
07:11:48,240 INFO JobControl util.MapRedUtil - Total input paths to process : 1
07:11:48,241 INFO JobControl util.MapRedUtil - Total input paths (combined) to process : 1
07:11:48,288 INFO Thread-591 mapred.LocalJobRunner - Waiting for map tasks
07:11:48,288 INFO pool-4-thread-1 mapred.LocalJobRunner - Starting task: attempt_local1888848137_0004_m_000000_0
07:11:48,301 INFO pool-4-thread-1 mapred.Task - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3b769b4a
07:11:48,304 INFO pool-4-thread-1 mapred.MapTask - Processing split: Number of splits :1
Total Length = 3045
Input split[0]:
Length = 3045
Locations:
-----------------------
07:11:48,308 INFO pool-4-thread-1 mapReduceLayer.PigRecordReader - Current split being processed file:/data/bamboo-home/xml-data/build-dir/SPRINGDATAHADOOP-HDP13CENTOS0-JOB1/src/test/resources/logs/apache_access.log:0+3045
07:11:48,319 INFO pool-4-thread-1 data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
07:11:48,325 INFO pool-4-thread-1 mapReduceLayer.PigMapOnly$Map - Aliases being processed per job phase (AliasName[line,offset]): M: A[1,4],B[-1,-1] C: R:
07:11:48,327 INFO pool-4-thread-1 mapred.Task - Task:attempt_local1888848137_0004_m_000000_0 is done. And is in the process of commiting
07:11:48,330 INFO pool-4-thread-1 mapred.LocalJobRunner -
07:11:48,330 INFO pool-4-thread-1 mapred.Task - Task attempt_local1888848137_0004_m_000000_0 is allowed to commit now
07:11:48,333 INFO pool-4-thread-1 output.FileOutputCommitter - Saved output of task 'attempt_local1888848137_0004_m_000000_0' to file:/tmp/temp922799163/tmp-1170206174
07:11:48,335 INFO pool-4-thread-1 mapred.LocalJobRunner -
07:11:48,335 INFO pool-4-thread-1 mapred.Task - Task 'attempt_local1888848137_0004_m_000000_0' done.
07:11:48,335 INFO pool-4-thread-1 mapred.LocalJobRunner - Finishing task: attempt_local1888848137_0004_m_000000_0
07:11:48,336 INFO Thread-591 mapred.LocalJobRunner - Map task executor complete.
07:11:48,723 INFO Test worker mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local1888848137_0004
07:11:48,723 INFO Test worker mapReduceLayer.MapReduceLauncher - Processing aliases A,B
07:11:48,723 INFO Test worker mapReduceLayer.MapReduceLauncher - detailed locations: M: A[1,4],B[-1,-1] C: R:
07:11:48,729 WARN Test worker pigstats.PigStatsUtil - Failed to get RunningJob for job job_local1888848137_0004
07:11:48,729 INFO Test worker mapReduceLayer.MapReduceLauncher - 100% complete
07:11:48,729 INFO Test worker pigstats.SimplePigStats - Detected Local mode. Stats reported below may be incomplete
07:11:48,730 INFO Test worker pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
1.2.0 0.11.0 bamboo 2013-09-05 07:11:48 2013-09-05 07:11:48 UNKNOWN
Success!
Job Stats (time in seconds):
JobId Alias Feature Outputs
job_local1888848137_0004 A,B MAP_ONLY file:/tmp/temp922799163/tmp-1170206174,
Input(s):
Successfully read records from: "file:///data/bamboo-home/xml-data/build-dir/SPRINGDATAHADOOP-HDP13CENTOS0-JOB1/src/test/resources/logs/apache_access.log"
Output(s):
Successfully stored records in: "file:/tmp/temp922799163/tmp-1170206174"
Job DAG:
job_local1888848137_0004
07:11:48,731 INFO Test worker mapReduceLayer.MapReduceLauncher - Success!
07:11:48,732 WARN Test worker data.SchemaTupleBackend - SchemaTupleBackend has already been initialized
07:11:48,734 INFO Test worker input.FileInputFormat - Total input paths to process : 1
07:11:48,734 INFO Test worker util.MapRedUtil - Total input paths to process : 1
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET / HTTP/1.1" 500 606 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:53:10 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/1.0" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /style.css HTTP/1.1" 200 614 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /img/pti-round.jpg HTTP/1.1" 200 17524 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:21 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET / HTTP/1.1" 200 34 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET /favicon.ico HTTP/1.1" 200 11514 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:53 +0300] "GET /cgi/pti.pl HTTP/1.1" 500 617 "http:/contact.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/0.9" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:27 +0300] "GET / HTTP/1.1" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:34 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:45 +0300] "GET /talks/Fundamentals/read-excel-file.html HTTP/1.1" 404 311 "http://pti.local/unix_sysadmin.html" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
07:11:48,760 INFO Test worker support.SimpleJobLauncher - Job: [FlowJob: [name=mainJob]] completed with the following parameters: [{}] and the following status: [COMPLETED]