Scala Common Enrich: support IPv6 addresses in IAB enrichment
See original GitHub issueWe’re seeing following bad row produced by IAB enrichment.
"errors":[{"level":"error","message":"Unexpected error processing events: java.lang.IllegalArgumentException: Could not parse [Redacted IPv6]\n\tat org.apache.commons.net.util.SubnetUtils.toInteger(SubnetUtils.java:287)\n\tat org.apache.commons.net.util.SubnetUtils.access$400(SubnetUtils.java:27)\n\tat org.apache.commons.net.util.SubnetUtils$SubnetInfo.isInRange(SubnetUtils.java:125)\n\tat com.snowplowanalytics.iab.spidersandrobotsclient.lib.internal.IpRanges.belong(IpRanges.java:78)\n\tat com.snowplowanalytics.iab.spidersandrobotsclient.IabClient.checkAt(IabClient.java:80)\n\tat com.snowplowanalytics.snowplow.enrich.common.enrichments.registry.IabEnrichment.performCheck(IabEnrichment.scala:198)\n\tat com.snowplowanalytics.snowplow.enrich.common.enrichments.registry.IabEnrichment.getIab(IabEnrichment.scala:233)\n\tat com.snowplowanalytics.snowplow.enrich.common.enrichments.registry.IabEnrichment.getIabContext(IabEnrichment.scala:218)\n\tat com.snowplowanalytics.snowplow.enrich.common.enrichments.EnrichmentManager$.enrichEvent(EnrichmentManager.scala:286)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3.apply(EtlPipeline.scala:92)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3.apply(EtlPipeline.scala:91)\n\tat scalaz.NonEmptyList$class.map(NonEmptyList.scala:23)\n\tat scalaz.NonEmptyListFunctions$$anon$4.map(NonEmptyList.scala:207)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2.apply(EtlPipeline.scala:91)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2.apply(EtlPipeline.scala:88)\n\tat scalaz.Validation$class.map(Validation.scala:112)\n\tat scalaz.Success.map(Validation.scala:345)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1.apply(EtlPipeline.scala:88)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1.apply(EtlPipeline.scala:85)\n\tat scala.Option.map(Option.scala:146)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1.apply(EtlPipeline.scala:85)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1.apply(EtlPipeline.scala:82)\n\tat scalaz.Validation$class.map(Validation.scala:112)\n\tat scalaz.Success.map(Validation.scala:345)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$.processEvents(EtlPipeline.scala:82)\n\tat com.snowplowanalytics.snowplow.enrich.beam.Enrich$.com$snowplowanalytics$snowplow$enrich$beam$Enrich$$enrich(Enrich.scala:204)\n\tat com.snowplowanalytics.snowplow.enrich.beam.Enrich$$anonfun$enrichEvents$1$$anonfun$12.apply(Enrich.scala:139)\n\tat com.snowplowanalytics.snowplow.enrich.beam.Enrich$$anonfun$enrichEvents$1$$anonfun$12.apply(Enrich.scala:139)\n\tat com.snowplowanalytics.snowplow.enrich.beam.utils$.timeMs(utils.scala:123)\n\tat com.snowplowanalytics.snowplow.enrich.beam.Enrich$$anonfun$enrichEvents$1.apply(Enrich.scala:138)\n\tat com.snowplowanalytics.snowplow.enrich.beam.Enrich$$anonfun$enrichEvents$1.apply(Enrich.scala:135)\n\tat com.spotify.scio.util.Functions$$anon$7.processElement(Functions.scala:145)\n\tat com.spotify.scio.util.Functions$$anon$7$DoFnInvoker.invokeProcessElement(Unknown Source)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:149)\n\tat com.google.cloud.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:323)\n\tat com.google.cloud.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:43)\n\tat com.google.cloud.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:48)\n\tat com.google.cloud.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:271)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:219)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.access$700(SimpleDoFnRunner.java:69)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:517)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:505)\n\tat com.spotify.scio.util.Functions$$anon$7.processElement(Functions.scala:145)\n\tat com.spotify.scio.util.Functions$$anon$7$DoFnInvoker.invokeProcessElement(Unknown Source)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:149)\n\tat com.google.cloud.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:323)\n\tat com.google.cloud.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:43)\n\tat com.google.cloud.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:48)\n\tat com.google.cloud.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:271)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:219)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.access$700(SimpleDoFnRunner.java:69)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:517)\n\tat org.apache.beam.sdk.transforms.DoFnOutputReceivers$WindowedContextOutputReceiver.output(DoFnOutputReceivers.java:71)\n\tat org.apache.beam.sdk.transforms.MapElements$1.processElement(MapElements.java:128)\n\tat org.apache.beam.sdk.transforms.MapElements$1$DoFnInvoker.invokeProcessElement(Unknown Source)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:149)\n\tat com.google.cloud.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:323)\n\tat com.google.cloud.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:43)\n\tat com.google.cloud.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:48)\n\tat com.google.cloud.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:200)\n\tat com.google.cloud.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:158)\n\tat com.google.cloud.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:75)\n\tat com.google.cloud.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1227)\n\tat com.google.cloud.dataflow.worker.StreamingDataflowWorker.access$1000(StreamingDataflowWorker.java:135)\n\tat com.google.cloud.dataflow.worker.StreamingDataflowWorker$6.run(StreamingDataflowWorker.java:966)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat java.lang.Thread.run(Thread.java:745)\n"}],"failure_tstamp":"2019-01-21T13:00:24.222Z"}
Issue Analytics
- State:
- Created 5 years ago
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Snowplow R114 released with new adapter, enrichment and ...
The feature has been added to Scala Common Enrich but can be used only in ... IAB Bots and Spiders enrichment skipped for...
Read more >We are no longer doing umbrella Snowplow releases and are ...
A new version of the Snowplow Common Enrich can be found on Maven repository. The schema for the configuration of the enrichment has...
Read more >Untitled
Qualcare claims address, 1st december 2012 balika vadhu, Work group costumes, Abati arabic, ... Fate stay fanfic recs, Hopcoms, Cisco 3550 ip subnet-zero, ......
Read more >IoT-From Research and Innovation to Market Deployment ...
All of these things have an IP address and can be tracked. ... for new industries to emerge and to deliver enriched and...
Read more >vocab.txt - Hugging Face
... arange ##address mis mom ##ral ##status activity png ##aw supports screen ... uncomment earthquake enriched ##ymbols grandparent bootstraps amendment ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
/cc @colmsnowplow
Moved to https://github.com/snowplow/enrich/issues/13