SemanticScuttle - klotz.me

Tags: hadoop*

The open source, distributed, parallel computation framework developed by Doug Cutting and Mike Cafarella and based on functional programming operations Map and Reduce, as described in the Google MapReduce paper.

0 bookmark(s) - Sort by: Date ↓ / Title /

31. Dealing with Corrupt Files in Hadoop

    2017-05-24 Tags: hadoop, gz, corruption by klotz
32. Reading Compressed Data With Spark Using Unknown File Extensions - Mutable Ideas

    val conf = new SparkConf()
    
    // Custom Codec that process .gz.tmp extensions as a common Gzip format
    conf.set("spark.hadoop.io.compression.codecs", "smx.ananke.spark.util.codecs.TmpGzipCodec")
    
    val sc = new SparkContext(conf)
    
    val data = sc.textFile("s3n://my-data-bucket/2015/09/21/13/*")

    2017-05-24 Tags: spark, compression, hadoop by klotz
33. Spark job to read gzip files, ignoring corrupted files

    2017-05-23 Tags: spark, gzip, hadoop by klotz
34. Spark to parse Weblogs text files and write output to Parquet format - Hortonworks

    2017-05-04 Tags: apache, zappelin, hortonworks, spark, logs, pyspark, hadoop by klotz
35. Jowanza Joseph — Creating A Spark Server For Every Job With Livy

    2017-04-20 Tags: spark, livy, hadoop by klotz
36. Hive on Spark: Getting Started - Apache Hive - Apache Software Foundation

    2017-04-10 Tags: spark, hadoop, yarn by klotz
37. Getting Started — Luigi 2.6.1 documentation

    2017-04-10 Tags: luigi, hadoop, orchestration by klotz
38. twitter/chill: Scala extensions for the Kryo serialization library

    2017-04-04 Tags: scala, hadoop, spark, kryo, serialization, java, performance by klotz
39. Avoiding Full GCs in Apache HBase with MemStore-Local Allocation Buffers: Part 1 - Cloudera Engineering Blog

    2017-04-04 Tags: hbase, performance, hadoop, java, gc, cloudera by klotz
40. Tuning Spark Applications

    2017-03-09 Tags: spark, performance, tuning, cloudera, hadoop by klotz

Top of the page

First / Previous / Next / Last / Page 4 of 0