07:11:48,767 INFO Test worker context.TestContextManager - @TestExecutionListeners is not present for class [class org.springframework.data.hadoop.pig.PigTest]: using defaults.
07:11:48,770 INFO Test worker xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/pig/basic.xml]
07:11:48,797 INFO Test worker xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/hadoop-ctx.xml]
07:11:48,823 INFO Test worker support.GenericApplicationContext - Refreshing org.springframework.context.support.GenericApplicationContext@64c6e290: startup date [Thu Sep 05 07:11:48 PDT 2013]; root of context hierarchy
07:11:48,830 INFO Test worker config.PropertyPlaceholderConfigurer - Loading properties file from class path resource [test.properties]
07:11:48,835 INFO Test worker support.DefaultListableBeanFactory - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@1ee2a54d: defining beans [ppc,hadoopFs,hadoopResourceLoader,hadoopConfiguration,cfg-init,fs-init,rl-init,org.springframework.data.hadoop.scripting.HdfsScriptRunner#0,org.springframework.data.hadoop.scripting.HdfsScriptRunner#1,pig-ctx,pig-raw,pigTemplate,foo,pigFactory,nop-script,pig-scripts,props,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor]; root of factory hierarchy
07:11:48,920 INFO Test worker config.PropertiesFactoryBean - Loading properties file from class path resource [props.properties]
07:11:48,921 INFO Test worker config.PropertiesFactoryBean - Loading properties file from class path resource [cfg-1.properties]
07:11:48,922 INFO Test worker config.PropertiesFactoryBean - Loading properties file from class path resource [cfg-2.properties]
07:11:48,986 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:11:48,986 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:11:49,025 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:11:49,026 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:11:49,093 INFO Test worker pigstats.ScriptState - Pig features used in the script: UNKNOWN
07:11:49,095 INFO Test worker rules.ColumnPruneVisitor - Columns pruned for A: $1
07:11:49,099 INFO Test worker mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
07:11:49,100 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
07:11:49,100 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
07:11:49,102 INFO Test worker pigstats.ScriptState - Pig script settings are added to the job
07:11:49,103 INFO Test worker mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
07:11:49,103 INFO Test worker mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
07:11:49,105 INFO Test worker mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=3045
07:11:49,105 INFO Test worker mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
07:11:49,105 INFO Test worker mapReduceLayer.JobControlCompiler - creating jar file Job6167983269790373030.jar
07:11:52,287 INFO Test worker mapReduceLayer.JobControlCompiler - jar file Job6167983269790373030.jar created
07:11:52,290 INFO Test worker mapReduceLayer.JobControlCompiler - Setting up single store job
07:11:52,291 INFO Test worker data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
07:11:52,291 INFO Test worker data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
07:11:52,291 INFO Test worker data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
07:11:52,294 INFO Test worker mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
07:11:52,298 WARN JobControl mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
07:11:52,353 INFO JobControl input.FileInputFormat - Total input paths to process : 1
07:11:52,353 INFO JobControl util.MapRedUtil - Total input paths to process : 1
07:11:52,354 INFO JobControl util.MapRedUtil - Total input paths (combined) to process : 1
07:11:52,795 INFO Test worker mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201308191125_0574
07:11:52,795 INFO Test worker mapReduceLayer.MapReduceLauncher - Processing aliases A,B
07:11:52,795 INFO Test worker mapReduceLayer.MapReduceLauncher - detailed locations: M: A[1,4],B[-1,-1] C: R:
07:11:52,795 INFO Test worker mapReduceLayer.MapReduceLauncher - More information at: http://w1-kodiak-hd023:50030/jobdetails.jsp?jobid=job_201308191125_0574
07:11:52,800 INFO Test worker mapReduceLayer.MapReduceLauncher - 0% complete
07:11:58,322 INFO Test worker mapReduceLayer.MapReduceLauncher - 50% complete
07:12:02,854 INFO Test worker mapReduceLayer.MapReduceLauncher - 100% complete
07:12:02,855 INFO Test worker pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
1.2.0 0.11.0 bamboo 2013-09-05 07:11:49 2013-09-05 07:12:02 UNKNOWN
Success!
Job Stats (time in seconds):
JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
job_201308191125_0574 1 0 2 2 2 2 0 0 0 0 A,B MAP_ONLY hdfs://w1-kodiak-hd023:8020/tmp/temp518875250/tmp1797124209,
Input(s):
Successfully read 16 records (3445 bytes) from: "hdfs://w1-kodiak-hd023:8020/user/bamboo/src/test/resources/logs/apache_access.log"
Output(s):
Successfully stored 16 records (3141 bytes) in: "hdfs://w1-kodiak-hd023:8020/tmp/temp518875250/tmp1797124209"
Counters:
Total records written : 16
Total bytes written : 3141
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_201308191125_0574
07:12:02,859 INFO Test worker mapReduceLayer.MapReduceLauncher - Success!
07:12:02,860 INFO Test worker data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
07:12:02,864 INFO Test worker input.FileInputFormat - Total input paths to process : 1
07:12:02,865 INFO Test worker util.MapRedUtil - Total input paths to process : 1
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET / HTTP/1.1" 500 606 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:53:10 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/1.0" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /style.css HTTP/1.1" 200 614 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /img/pti-round.jpg HTTP/1.1" 200 17524 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:21 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET / HTTP/1.1" 200 34 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET /favicon.ico HTTP/1.1" 200 11514 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:53 +0300] "GET /cgi/pti.pl HTTP/1.1" 500 617 "http:/contact.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/0.9" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:27 +0300] "GET / HTTP/1.1" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:34 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:45 +0300] "GET /talks/Fundamentals/read-excel-file.html HTTP/1.1" 404 311 "http://pti.local/unix_sysadmin.html" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
07:12:02,926 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:02,926 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:12:02,966 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:02,966 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:12:03,027 INFO Test worker pigstats.ScriptState - Pig features used in the script: UNKNOWN
07:12:03,029 INFO Test worker rules.ColumnPruneVisitor - Columns pruned for A: $1
07:12:03,033 INFO Test worker mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
07:12:03,035 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
07:12:03,035 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
07:12:03,036 INFO Test worker pigstats.ScriptState - Pig script settings are added to the job
07:12:03,037 INFO Test worker mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
07:12:03,038 INFO Test worker mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
07:12:03,039 INFO Test worker mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=3045
07:12:03,039 INFO Test worker mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
07:12:03,040 INFO Test worker mapReduceLayer.JobControlCompiler - creating jar file Job4586948304246378088.jar
07:12:05,353 INFO Test worker mapReduceLayer.JobControlCompiler - jar file Job4586948304246378088.jar created
07:12:05,355 INFO Test worker mapReduceLayer.JobControlCompiler - Setting up single store job
07:12:05,356 INFO Test worker data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
07:12:05,356 INFO Test worker data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
07:12:05,356 INFO Test worker data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
07:12:05,359 INFO Test worker mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
07:12:05,362 WARN JobControl mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
07:12:05,427 INFO JobControl input.FileInputFormat - Total input paths to process : 1
07:12:05,427 INFO JobControl util.MapRedUtil - Total input paths to process : 1
07:12:05,428 INFO JobControl util.MapRedUtil - Total input paths (combined) to process : 1
Called script
07:12:05,677 INFO myScheduler-3 mapreduce.JobRunner - Starting job [custom-jar-job]
07:12:05,696 INFO myScheduler-3 mapred.JobClient - Cleaning up the staging area hdfs://w1-kodiak-hd023:8020/app/hadoop/tmp/mapred/staging/bamboo/.staging/job_201308191125_0576
07:12:05,697 ERROR myScheduler-3 security.UserGroupInformation - PriviledgedActionException as:bamboo cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /ide-test/runner/output-4 already exists
07:12:05,697 WARN myScheduler-3 mapreduce.JobRunner - Cannot start job [custom-jar-job]
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /ide-test/runner/output-4 already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:973)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
at org.springframework.data.hadoop.mapreduce.JobExecutor$2.run(JobExecutor.java:201)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
at org.springframework.data.hadoop.mapreduce.JobExecutor.startJobs(JobExecutor.java:172)
at org.springframework.data.hadoop.mapreduce.JobExecutor.startJobs(JobExecutor.java:164)
at org.springframework.data.hadoop.mapreduce.JobRunner.call(JobRunner.java:52)
at org.springframework.data.hadoop.mapreduce.JobRunner.call(JobRunner.java:33)
at org.springframework.data.hadoop.mapreduce.JobRunner.invoke(JobRunner.java:88)
at org.springframework.data.hadoop.mapreduce.JobRunner.call(JobRunner.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:273)
at org.springframework.scheduling.support.MethodInvokingRunnable.run(MethodInvokingRunnable.java:65)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:51)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
07:12:05,698 ERROR myScheduler-3 support.MethodInvokingRunnable - Invocation of method 'call' on target class [class org.springframework.data.hadoop.mapreduce.JobRunner] failed
java.lang.IllegalStateException: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /ide-test/runner/output-4 already exists
at org.springframework.data.hadoop.mapreduce.JobExecutor$2.run(JobExecutor.java:213)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
at org.springframework.data.hadoop.mapreduce.JobExecutor.startJobs(JobExecutor.java:172)
at org.springframework.data.hadoop.mapreduce.JobExecutor.startJobs(JobExecutor.java:164)
at org.springframework.data.hadoop.mapreduce.JobRunner.call(JobRunner.java:52)
at org.springframework.data.hadoop.mapreduce.JobRunner.call(JobRunner.java:33)
at org.springframework.data.hadoop.mapreduce.JobRunner.invoke(JobRunner.java:88)
at org.springframework.data.hadoop.mapreduce.JobRunner.call(JobRunner.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:273)
at org.springframework.scheduling.support.MethodInvokingRunnable.run(MethodInvokingRunnable.java:65)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:51)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
Caused by: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /ide-test/runner/output-4 already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:973)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
at org.springframework.data.hadoop.mapreduce.JobExecutor$2.run(JobExecutor.java:201)
... 22 more
07:12:05,859 INFO Test worker mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201308191125_0575
07:12:05,859 INFO Test worker mapReduceLayer.MapReduceLauncher - Processing aliases A,B
07:12:05,859 INFO Test worker mapReduceLayer.MapReduceLauncher - detailed locations: M: A[1,4],B[-1,-1] C: R:
07:12:05,859 INFO Test worker mapReduceLayer.MapReduceLauncher - More information at: http://w1-kodiak-hd023:50030/jobdetails.jsp?jobid=job_201308191125_0575
07:12:05,862 INFO Test worker mapReduceLayer.MapReduceLauncher - 0% complete
07:12:10,876 INFO Test worker mapReduceLayer.MapReduceLauncher - 50% complete
07:12:15,901 INFO Test worker mapReduceLayer.MapReduceLauncher - 100% complete
07:12:15,902 INFO Test worker pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
1.2.0 0.11.0 bamboo 2013-09-05 07:12:03 2013-09-05 07:12:15 UNKNOWN
Success!
Job Stats (time in seconds):
JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
job_201308191125_0575 1 0 2 2 2 2 0 0 0 0 A,B MAP_ONLY hdfs://w1-kodiak-hd023:8020/tmp/temp518875250/tmp-1269767314,
Input(s):
Successfully read 16 records (3445 bytes) from: "hdfs://w1-kodiak-hd023:8020/user/bamboo/src/test/resources/logs/apache_access.log"
Output(s):
Successfully stored 16 records (3141 bytes) in: "hdfs://w1-kodiak-hd023:8020/tmp/temp518875250/tmp-1269767314"
Counters:
Total records written : 16
Total bytes written : 3141
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_201308191125_0575
07:12:15,905 INFO Test worker mapReduceLayer.MapReduceLauncher - Success!
07:12:15,905 INFO Test worker data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
07:12:15,917 INFO Test worker input.FileInputFormat - Total input paths to process : 1
07:12:15,917 INFO Test worker util.MapRedUtil - Total input paths to process : 1
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET / HTTP/1.1" 500 606 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:53:10 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/1.0" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /style.css HTTP/1.1" 200 614 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /img/pti-round.jpg HTTP/1.1" 200 17524 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:21 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET / HTTP/1.1" 200 34 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET /favicon.ico HTTP/1.1" 200 11514 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:53 +0300] "GET /cgi/pti.pl HTTP/1.1" 500 617 "http:/contact.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/0.9" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:27 +0300] "GET / HTTP/1.1" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:34 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:45 +0300] "GET /talks/Fundamentals/read-excel-file.html HTTP/1.1" 404 311 "http://pti.local/unix_sysadmin.html" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
07:12:15,991 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:15,992 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:12:16,030 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:16,031 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:12:16,070 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:16,071 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:12:16,129 INFO Test worker pigstats.ScriptState - Pig features used in the script: UNKNOWN
07:12:16,131 INFO Test worker rules.ColumnPruneVisitor - Columns pruned for A: $1
07:12:16,135 INFO Test worker mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
07:12:16,136 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
07:12:16,136 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
07:12:16,138 INFO Test worker pigstats.ScriptState - Pig script settings are added to the job
07:12:16,139 INFO Test worker mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
07:12:16,139 INFO Test worker mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
07:12:16,141 INFO Test worker mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=3045
07:12:16,141 INFO Test worker mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
07:12:16,142 INFO Test worker mapReduceLayer.JobControlCompiler - creating jar file Job3431065593660411230.jar
07:12:18,587 INFO Test worker mapReduceLayer.JobControlCompiler - jar file Job3431065593660411230.jar created
07:12:18,590 INFO Test worker mapReduceLayer.JobControlCompiler - Setting up single store job
07:12:18,590 INFO Test worker data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
07:12:18,590 INFO Test worker data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
07:12:18,590 INFO Test worker data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
07:12:18,593 INFO Test worker mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
07:12:18,596 WARN JobControl mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
07:12:18,647 INFO JobControl input.FileInputFormat - Total input paths to process : 1
07:12:18,648 INFO JobControl util.MapRedUtil - Total input paths to process : 1
07:12:18,648 INFO JobControl util.MapRedUtil - Total input paths (combined) to process : 1
07:12:19,093 INFO Test worker mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201308191125_0577
07:12:19,094 INFO Test worker mapReduceLayer.MapReduceLauncher - Processing aliases A,B
07:12:19,094 INFO Test worker mapReduceLayer.MapReduceLauncher - detailed locations: M: A[1,4],B[-1,-1] C: R:
07:12:19,094 INFO Test worker mapReduceLayer.MapReduceLauncher - More information at: http://w1-kodiak-hd023:50030/jobdetails.jsp?jobid=job_201308191125_0577
07:12:19,099 INFO Test worker mapReduceLayer.MapReduceLauncher - 0% complete
07:12:24,615 INFO Test worker mapReduceLayer.MapReduceLauncher - 50% complete
07:12:29,137 INFO Test worker mapReduceLayer.MapReduceLauncher - 100% complete
07:12:29,138 INFO Test worker pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
1.2.0 0.11.0 bamboo 2013-09-05 07:12:16 2013-09-05 07:12:29 UNKNOWN
Success!
Job Stats (time in seconds):
JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
job_201308191125_0577 1 0 3 3 3 3 0 0 0 0 A,B MAP_ONLY hdfs://w1-kodiak-hd023:8020/tmp/temp518875250/tmp-341785253,
Input(s):
Successfully read 16 records (3445 bytes) from: "hdfs://w1-kodiak-hd023:8020/user/bamboo/src/test/resources/logs/apache_access.log"
Output(s):
Successfully stored 16 records (3141 bytes) in: "hdfs://w1-kodiak-hd023:8020/tmp/temp518875250/tmp-341785253"
Counters:
Total records written : 16
Total bytes written : 3141
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_201308191125_0577
07:12:29,141 INFO Test worker mapReduceLayer.MapReduceLauncher - Success!
07:12:29,142 INFO Test worker data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
07:12:29,146 INFO Test worker input.FileInputFormat - Total input paths to process : 1
07:12:29,146 INFO Test worker util.MapRedUtil - Total input paths to process : 1
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET / HTTP/1.1" 500 606 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:53:10 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/1.0" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /style.css HTTP/1.1" 200 614 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /img/pti-round.jpg HTTP/1.1" 200 17524 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:21 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET / HTTP/1.1" 200 34 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET /favicon.ico HTTP/1.1" 200 11514 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:53 +0300] "GET /cgi/pti.pl HTTP/1.1" 500 617 "http:/contact.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/0.9" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:27 +0300] "GET / HTTP/1.1" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:34 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:45 +0300] "GET /talks/Fundamentals/read-excel-file.html HTTP/1.1" 404 311 "http://pti.local/unix_sysadmin.html" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
07:12:29,214 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:29,215 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:12:29,303 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:29,304 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:12:29,344 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:29,345 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:12:29,405 INFO Test worker pigstats.ScriptState - Pig features used in the script: UNKNOWN
07:12:29,408 INFO Test worker rules.ColumnPruneVisitor - Columns pruned for A: $1
07:12:29,412 INFO Test worker mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
07:12:29,413 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
07:12:29,413 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
07:12:29,415 INFO Test worker pigstats.ScriptState - Pig script settings are added to the job
07:12:29,416 INFO Test worker mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
07:12:29,416 INFO Test worker mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
07:12:29,418 INFO Test worker mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=3045
07:12:29,419 INFO Test worker mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
07:12:29,419 INFO Test worker mapReduceLayer.JobControlCompiler - creating jar file Job4856635542341170305.jar
07:12:32,026 INFO Test worker mapReduceLayer.JobControlCompiler - jar file Job4856635542341170305.jar created
07:12:32,028 INFO Test worker mapReduceLayer.JobControlCompiler - Setting up single store job
07:12:32,029 INFO Test worker data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
07:12:32,029 INFO Test worker data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
07:12:32,029 INFO Test worker data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
07:12:32,032 INFO Test worker mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
07:12:32,036 WARN JobControl mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
07:12:32,076 INFO JobControl input.FileInputFormat - Total input paths to process : 1
07:12:32,076 INFO JobControl util.MapRedUtil - Total input paths to process : 1
07:12:32,078 INFO JobControl util.MapRedUtil - Total input paths (combined) to process : 1
07:12:32,532 INFO Test worker mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201308191125_0578
07:12:32,532 INFO Test worker mapReduceLayer.MapReduceLauncher - Processing aliases A,B
07:12:32,532 INFO Test worker mapReduceLayer.MapReduceLauncher - detailed locations: M: A[1,4],B[-1,-1] C: R:
07:12:32,532 INFO Test worker mapReduceLayer.MapReduceLauncher - More information at: http://w1-kodiak-hd023:50030/jobdetails.jsp?jobid=job_201308191125_0578
07:12:32,535 INFO Test worker mapReduceLayer.MapReduceLauncher - 0% complete
07:12:36,544 INFO Test worker mapReduceLayer.MapReduceLauncher - 50% complete
07:12:42,572 INFO Test worker mapReduceLayer.MapReduceLauncher - 100% complete
07:12:42,573 INFO Test worker pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
1.2.0 0.11.0 bamboo 2013-09-05 07:12:29 2013-09-05 07:12:42 UNKNOWN
Success!
Job Stats (time in seconds):
JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
job_201308191125_0578 1 0 2 2 2 2 0 0 0 0 A,B MAP_ONLY hdfs://w1-kodiak-hd023:8020/tmp/temp1904969845/tmp217304311,
Input(s):
Successfully read 16 records (3445 bytes) from: "hdfs://w1-kodiak-hd023:8020/user/bamboo/src/test/resources/logs/apache_access.log"
Output(s):
Successfully stored 16 records (3141 bytes) in: "hdfs://w1-kodiak-hd023:8020/tmp/temp1904969845/tmp217304311"
Counters:
Total records written : 16
Total bytes written : 3141
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_201308191125_0578
07:12:42,579 INFO Test worker mapReduceLayer.MapReduceLauncher - Success!
07:12:42,580 INFO Test worker data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
07:12:42,585 INFO Test worker input.FileInputFormat - Total input paths to process : 1
07:12:42,585 INFO Test worker util.MapRedUtil - Total input paths to process : 1
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET / HTTP/1.1" 500 606 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:53:10 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/1.0" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /style.css HTTP/1.1" 200 614 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /img/pti-round.jpg HTTP/1.1" 200 17524 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:21 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET / HTTP/1.1" 200 34 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET /favicon.ico HTTP/1.1" 200 11514 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:53 +0300] "GET /cgi/pti.pl HTTP/1.1" 500 617 "http:/contact.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/0.9" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:27 +0300] "GET / HTTP/1.1" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:34 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:45 +0300] "GET /talks/Fundamentals/read-excel-file.html HTTP/1.1" 404 311 "http://pti.local/unix_sysadmin.html" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
1
07:12:42,651 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:42,652 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:12:42,690 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:42,691 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
07:12:42,749 INFO Test worker pigstats.ScriptState - Pig features used in the script: UNKNOWN
07:12:42,751 INFO Test worker rules.ColumnPruneVisitor - Columns pruned for A: $1
07:12:42,755 INFO Test worker mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
07:12:42,756 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
07:12:42,756 INFO Test worker mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
07:12:42,758 INFO Test worker pigstats.ScriptState - Pig script settings are added to the job
07:12:42,759 INFO Test worker mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
07:12:42,759 INFO Test worker mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
07:12:42,761 INFO Test worker mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=3045
07:12:42,761 INFO Test worker mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
07:12:42,761 INFO Test worker mapReduceLayer.JobControlCompiler - creating jar file Job1608601345296056161.jar
07:12:45,152 INFO Test worker mapReduceLayer.JobControlCompiler - jar file Job1608601345296056161.jar created
07:12:45,154 INFO Test worker mapReduceLayer.JobControlCompiler - Setting up single store job
07:12:45,154 INFO Test worker data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
07:12:45,154 INFO Test worker data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
07:12:45,155 INFO Test worker data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
07:12:45,158 INFO Test worker mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
07:12:45,161 WARN JobControl mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
07:12:45,204 INFO JobControl input.FileInputFormat - Total input paths to process : 1
07:12:45,204 INFO JobControl util.MapRedUtil - Total input paths to process : 1
07:12:45,205 INFO JobControl util.MapRedUtil - Total input paths (combined) to process : 1
07:12:45,659 INFO Test worker mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201308191125_0579
07:12:45,659 INFO Test worker mapReduceLayer.MapReduceLauncher - Processing aliases A,B
07:12:45,659 INFO Test worker mapReduceLayer.MapReduceLauncher - detailed locations: M: A[1,4],B[-1,-1] C: R:
07:12:45,659 INFO Test worker mapReduceLayer.MapReduceLauncher - More information at: http://w1-kodiak-hd023:50030/jobdetails.jsp?jobid=job_201308191125_0579
07:12:45,664 INFO Test worker mapReduceLayer.MapReduceLauncher - 0% complete
07:12:53,191 INFO Test worker mapReduceLayer.MapReduceLauncher - 50% complete
07:12:55,707 INFO Test worker mapReduceLayer.MapReduceLauncher - 100% complete
07:12:55,708 INFO Test worker pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
1.2.0 0.11.0 bamboo 2013-09-05 07:12:42 2013-09-05 07:12:55 UNKNOWN
Success!
Job Stats (time in seconds):
JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
job_201308191125_0579 1 0 3 3 3 3 0 0 0 0 A,B MAP_ONLY hdfs://w1-kodiak-hd023:8020/tmp/temp1904969845/tmp227058253,
Input(s):
Successfully read 16 records (3445 bytes) from: "hdfs://w1-kodiak-hd023:8020/user/bamboo/src/test/resources/logs/apache_access.log"
Output(s):
Successfully stored 16 records (3141 bytes) in: "hdfs://w1-kodiak-hd023:8020/tmp/temp1904969845/tmp227058253"
Counters:
Total records written : 16
Total bytes written : 3141
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_201308191125_0579
07:12:55,716 INFO Test worker mapReduceLayer.MapReduceLauncher - Success!
07:12:55,717 INFO Test worker data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
07:12:55,722 INFO Test worker input.FileInputFormat - Total input paths to process : 1
07:12:55,722 INFO Test worker util.MapRedUtil - Total input paths to process : 1
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET / HTTP/1.1" 500 606 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:39:11 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(139.12.0.2 - - [10/Apr/2007:10:40:54 +0300] "GET /favicon.ico HTTP/1.1" 200 766 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:53:10 +0300] "GET / HTTP/1.1" 500 612 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/1.0" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /style.css HTTP/1.1" 200 614 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET /img/pti-round.jpg HTTP/1.1" 200 17524 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:21 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET / HTTP/1.1" 200 34 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:51 +0300] "GET /favicon.ico HTTP/1.1" 200 11514 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:54:53 +0300] "GET /cgi/pti.pl HTTP/1.1" 500 617 "http:/contact.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(127.0.0.1 - - [10/Apr/2007:10:54:08 +0300] "GET / HTTP/0.9" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:27 +0300] "GET / HTTP/1.1" 200 3700 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:34 +0300] "GET /unix_sysadmin.html HTTP/1.1" 200 3880 "http://pti.local/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
(217.0.22.3 - - [10/Apr/2007:10:58:45 +0300] "GET /talks/Fundamentals/read-excel-file.html HTTP/1.1" 404 311 "http://pti.local/unix_sysadmin.html" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)")
07:12:55,786 INFO Test worker executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://w1-kodiak-hd023:8020
07:12:55,787 INFO Test worker executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: w1-kodiak-hd023:51130
[A, B]