hadoop - Hive CLI not populating table data (from Create Table as Select Query) while Hive Beeswax works fine -
when running "create table select" query on hive cli, table got created data not getting populated. when running same query on hive beeswax, getting target table created data populated in it.
here query:
hive -e ' create table table_validation select listing_id, city, area, expected_amount_inr, property_id, house_type, case when area_builtup_sqft null or area_builtup_sqft = 0 or area_builtup_sqft = " " plot_area else area_builtup_sqft end area_sqft, case when area_builtup_sqft null or area_builtup_sqft = 0 or area_builtup_sqft = " " expected_amount_inr/plot_area else expected_amount_inr/area_builtup_sqft end price_sqft,listing_state, case when house_type "apartment" "apartment" when house_type "plot" "plot" else "others" end property_type, case when house_type "plot" "na" when num_bedrooms between 1 , 1.9 1 when num_bedrooms between 2 , 2.9 2 when num_bedrooms between 3 , 3.9 3 when num_bedrooms >= 4 4 else num_bedrooms end number_bedrooms realestate_listing_main listing_type "rent" , added_on between '2015-02-01' , '2015-03-31' ' --database default; when running query, getting following result:
running hive query 0 2015-03-31 18:40:41,025 info [main] configuration.deprecation (configuration.java:warnonceifdeprecated(1011)) - mapred.input.dir.recursive deprecated. instead, use mapreduce.input.fileinputformat.input.dir.recursive 2015-03-31 18:40:41,030 info [main] configuration.deprecation (configuration.java:warnonceifdeprecated(1011)) - mapred.max.split.size deprecated. instead, use mapreduce.input.fileinputformat.split.maxsize 2015-03-31 18:40:41,030 info [main] configuration.deprecation (configuration.java:warnonceifdeprecated(1011)) - mapred.min.split.size deprecated. instead, use mapreduce.input.fileinputformat.split.minsize 2015-03-31 18:40:41,030 info [main] configuration.deprecation (configuration.java:warnonceifdeprecated(1011)) - mapred.min.split.size.per.rack deprecated. instead, use mapreduce.input.fileinputformat.split.minsize.per.rack 2015-03-31 18:40:41,031 info [main] configuration.deprecation (configuration.java:warnonceifdeprecated(1011)) - mapred.min.split.size.per.node deprecated. instead, use mapreduce.input.fileinputformat.split.minsize.per.node 2015-03-31 18:40:41,031 info [main] configuration.deprecation (configuration.java:warnonceifdeprecated(1011)) - mapred.reduce.tasks deprecated. instead, use mapreduce.job.reduces 2015-03-31 18:40:41,031 info [main] configuration.deprecation (configuration.java:warnonceifdeprecated(1011)) - mapred.reduce.tasks.speculative.execution deprecated. instead, use mapreduce.reduce.speculative 2015-03-31 18:40:41,336 warn [main] conf.hiveconf (hiveconf.java:initialize(1155)) - deprecated: configuration property hive.metastore.local no longer has effect. make sure provide valid value hive.metastore.uris if connecting remote metastore. logging initialized using configuration in jar:file:/usr/lib/hive/lib/hive-common-0.12.0-cdh5.1.2.jar!/hive-log4j.properties ok time taken: 0.621 seconds total mapreduce jobs = 3 launching job 1 out of 3 number of reduce tasks set 0 since there's no reduce operator starting job = job_1427789583342_0014, tracking url = http://ip-10-172-133-249.ap-southeast-1.compute.internal:8088/proxy/application_1427789583342_0014/ kill command = /usr/lib/hadoop/bin/hadoop job -kill job_1427789583342_0014 hadoop job information stage-1: number of mappers: 10; number of reducers: 0 2015-03-31 18:40:59,849 stage-1 map = 0%, reduce = 0% 2015-03-31 18:41:10,188 stage-1 map = 10%, reduce = 0%, cumulative cpu 5.86 sec 2015-03-31 18:41:11,219 stage-1 map = 10%, reduce = 0%, cumulative cpu 5.86 sec 2015-03-31 18:41:12,252 stage-1 map = 10%, reduce = 0%, cumulative cpu 5.86 sec 2015-03-31 18:41:13,289 stage-1 map = 10%, reduce = 0%, cumulative cpu 5.86 sec 2015-03-31 18:41:14,321 stage-1 map = 10%, reduce = 0%, cumulative cpu 5.86 sec 2015-03-31 18:41:15,357 stage-1 map = 10%, reduce = 0%, cumulative cpu 5.86 sec 2015-03-31 18:41:16,393 stage-1 map = 35%, reduce = 0%, cumulative cpu 39.78 sec 2015-03-31 18:41:17,428 stage-1 map = 40%, reduce = 0%, cumulative cpu 41.17 sec 2015-03-31 18:41:18,460 stage-1 map = 45%, reduce = 0%, cumulative cpu 43.26 sec 2015-03-31 18:41:19,499 stage-1 map = 67%, reduce = 0%, cumulative cpu 49.68 sec 2015-03-31 18:41:20,536 stage-1 map = 70%, reduce = 0%, cumulative cpu 50.49 sec 2015-03-31 18:41:21,569 stage-1 map = 80%, reduce = 0%, cumulative cpu 56.28 sec 2015-03-31 18:41:22,598 stage-1 map = 80%, reduce = 0%, cumulative cpu 56.28 sec 2015-03-31 18:41:23,627 stage-1 map = 80%, reduce = 0%, cumulative cpu 56.28 sec 2015-03-31 18:41:24,655 stage-1 map = 80%, reduce = 0%, cumulative cpu 56.28 sec 2015-03-31 18:41:25,684 stage-1 map = 80%, reduce = 0%, cumulative cpu 56.28 sec 2015-03-31 18:41:26,714 stage-1 map = 80%, reduce = 0%, cumulative cpu 56.28 sec 2015-03-31 18:41:27,743 stage-1 map = 80%, reduce = 0%, cumulative cpu 56.28 sec 2015-03-31 18:41:28,773 stage-1 map = 80%, reduce = 0%, cumulative cpu 56.28 sec 2015-03-31 18:41:29,803 stage-1 map = 85%, reduce = 0%, cumulative cpu 61.88 sec 2015-03-31 18:41:30,840 stage-1 map = 90%, reduce = 0%, cumulative cpu 63.8 sec 2015-03-31 18:41:31,872 stage-1 map = 90%, reduce = 0%, cumulative cpu 63.8 sec 2015-03-31 18:41:32,905 stage-1 map = 95%, reduce = 0%, cumulative cpu 69.86 sec 2015-03-31 18:41:33,935 stage-1 map = 100%, reduce = 0%, cumulative cpu 71.58 sec 2015-03-31 18:41:34,964 stage-1 map = 100%, reduce = 0%, cumulative cpu 71.58 sec mapreduce total cumulative cpu time: 1 minutes 11 seconds 580 msec ended job = job_1427789583342_0014 stage-4 selected condition resolver. stage-3 filtered out condition resolver. stage-5 filtered out condition resolver. moving data to: hdfs://ip-10-172-133-249.ap-southeast-1.compute.internal:8020/tmp/hive-root/hive_2015-03-31_18-40-42_689_38529489390850959-1/-ext-10001 moving data to: hdfs://ip-10-172-133-249.ap-southeast-1.compute.internal:8020/user/hive/warehouse/default.db/table_validation table default.table_validation stats: [num_partitions: 0, num_files: 10, num_rows: 0, total_size: 0, raw_data_size: 0] mapreduce jobs launched: job 0: map: 10 cumulative cpu: 71.58 sec hdfs read: 2635527679 hdfs write: 0 success total mapreduce cpu time spent: 1 minutes 11 seconds 580 msec ok time taken: 52.896 seconds it's not executing 2nd , 3rd job. when run query on hive beeswax, jobs getting executed , table created data.
please let me know missing? stuck on past 3 days.
got answer. need add serde.jar before running query hive can't recognize data without jar.
Comments
Post a Comment