SemanticScuttle - klotz.me

Tags: error* + gzip*

0 bookmark(s) - Sort by: Date ↓ / Title /

1. [CORE] [SPARK-6593] Provide option for HadoopRDD to skip corrupted files by tigerquoll · Pull Request #5250 · apache/spark

   sc.parallelize(fs.listStatus(...).map(_.getPath.toString)).map { pathStr =>
   val path = new Path(pathStr)
   val in = new GZIPInputStream(fs.open(path))
   try {
   in.read()
   } catch {
   case e: ZipException => fs.delete(path, false)
   } finally {
   in.close()
   }
   }

   2017-06-05 Tags: spark, gzip, error by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0