question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Scala Common Enrich: support IPv6 addresses in IAB enrichment

See original GitHub issue

We’re seeing following bad row produced by IAB enrichment.

"errors":[{"level":"error","message":"Unexpected error processing events: java.lang.IllegalArgumentException: Could not parse [Redacted IPv6]\n\tat org.apache.commons.net.util.SubnetUtils.toInteger(SubnetUtils.java:287)\n\tat org.apache.commons.net.util.SubnetUtils.access$400(SubnetUtils.java:27)\n\tat org.apache.commons.net.util.SubnetUtils$SubnetInfo.isInRange(SubnetUtils.java:125)\n\tat com.snowplowanalytics.iab.spidersandrobotsclient.lib.internal.IpRanges.belong(IpRanges.java:78)\n\tat com.snowplowanalytics.iab.spidersandrobotsclient.IabClient.checkAt(IabClient.java:80)\n\tat com.snowplowanalytics.snowplow.enrich.common.enrichments.registry.IabEnrichment.performCheck(IabEnrichment.scala:198)\n\tat com.snowplowanalytics.snowplow.enrich.common.enrichments.registry.IabEnrichment.getIab(IabEnrichment.scala:233)\n\tat com.snowplowanalytics.snowplow.enrich.common.enrichments.registry.IabEnrichment.getIabContext(IabEnrichment.scala:218)\n\tat com.snowplowanalytics.snowplow.enrich.common.enrichments.EnrichmentManager$.enrichEvent(EnrichmentManager.scala:286)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3.apply(EtlPipeline.scala:92)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3.apply(EtlPipeline.scala:91)\n\tat scalaz.NonEmptyList$class.map(NonEmptyList.scala:23)\n\tat scalaz.NonEmptyListFunctions$$anon$4.map(NonEmptyList.scala:207)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2.apply(EtlPipeline.scala:91)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2.apply(EtlPipeline.scala:88)\n\tat scalaz.Validation$class.map(Validation.scala:112)\n\tat scalaz.Success.map(Validation.scala:345)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1.apply(EtlPipeline.scala:88)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1$$anonfun$apply$1.apply(EtlPipeline.scala:85)\n\tat scala.Option.map(Option.scala:146)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1.apply(EtlPipeline.scala:85)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$$anonfun$1.apply(EtlPipeline.scala:82)\n\tat scalaz.Validation$class.map(Validation.scala:112)\n\tat scalaz.Success.map(Validation.scala:345)\n\tat com.snowplowanalytics.snowplow.enrich.common.EtlPipeline$.processEvents(EtlPipeline.scala:82)\n\tat com.snowplowanalytics.snowplow.enrich.beam.Enrich$.com$snowplowanalytics$snowplow$enrich$beam$Enrich$$enrich(Enrich.scala:204)\n\tat com.snowplowanalytics.snowplow.enrich.beam.Enrich$$anonfun$enrichEvents$1$$anonfun$12.apply(Enrich.scala:139)\n\tat com.snowplowanalytics.snowplow.enrich.beam.Enrich$$anonfun$enrichEvents$1$$anonfun$12.apply(Enrich.scala:139)\n\tat com.snowplowanalytics.snowplow.enrich.beam.utils$.timeMs(utils.scala:123)\n\tat com.snowplowanalytics.snowplow.enrich.beam.Enrich$$anonfun$enrichEvents$1.apply(Enrich.scala:138)\n\tat com.snowplowanalytics.snowplow.enrich.beam.Enrich$$anonfun$enrichEvents$1.apply(Enrich.scala:135)\n\tat com.spotify.scio.util.Functions$$anon$7.processElement(Functions.scala:145)\n\tat com.spotify.scio.util.Functions$$anon$7$DoFnInvoker.invokeProcessElement(Unknown Source)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:149)\n\tat com.google.cloud.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:323)\n\tat com.google.cloud.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:43)\n\tat com.google.cloud.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:48)\n\tat com.google.cloud.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:271)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:219)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.access$700(SimpleDoFnRunner.java:69)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:517)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:505)\n\tat com.spotify.scio.util.Functions$$anon$7.processElement(Functions.scala:145)\n\tat com.spotify.scio.util.Functions$$anon$7$DoFnInvoker.invokeProcessElement(Unknown Source)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:149)\n\tat com.google.cloud.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:323)\n\tat com.google.cloud.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:43)\n\tat com.google.cloud.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:48)\n\tat com.google.cloud.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:271)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:219)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.access$700(SimpleDoFnRunner.java:69)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:517)\n\tat org.apache.beam.sdk.transforms.DoFnOutputReceivers$WindowedContextOutputReceiver.output(DoFnOutputReceivers.java:71)\n\tat org.apache.beam.sdk.transforms.MapElements$1.processElement(MapElements.java:128)\n\tat org.apache.beam.sdk.transforms.MapElements$1$DoFnInvoker.invokeProcessElement(Unknown Source)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)\n\tat org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:149)\n\tat com.google.cloud.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:323)\n\tat com.google.cloud.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:43)\n\tat com.google.cloud.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:48)\n\tat com.google.cloud.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:200)\n\tat com.google.cloud.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:158)\n\tat com.google.cloud.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:75)\n\tat com.google.cloud.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1227)\n\tat com.google.cloud.dataflow.worker.StreamingDataflowWorker.access$1000(StreamingDataflowWorker.java:135)\n\tat com.google.cloud.dataflow.worker.StreamingDataflowWorker$6.run(StreamingDataflowWorker.java:966)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat java.lang.Thread.run(Thread.java:745)\n"}],"failure_tstamp":"2019-01-21T13:00:24.222Z"}

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
chuwycommented, Jan 21, 2019
0reactions
chuwycommented, Jun 19, 2020
Read more comments on GitHub >

github_iconTop Results From Across the Web

Snowplow R114 released with new adapter, enrichment and ...
The feature has been added to Scala Common Enrich but can be used only in ... IAB Bots and Spiders enrichment skipped for...
Read more >
We are no longer doing umbrella Snowplow releases and are ...
A new version of the Snowplow Common Enrich can be found on Maven repository. The schema for the configuration of the enrichment has...
Read more >
Untitled
Qualcare claims address, 1st december 2012 balika vadhu, Work group costumes, Abati arabic, ... Fate stay fanfic recs, Hopcoms, Cisco 3550 ip subnet-zero, ......
Read more >
IoT-From Research and Innovation to Market Deployment ...
All of these things have an IP address and can be tracked. ... for new industries to emerge and to deliver enriched and...
Read more >
vocab.txt - Hugging Face
... arange ##address mis mom ##ral ##status activity png ##aw supports screen ... uncomment earthquake enriched ##ymbols grandparent bootstraps amendment ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found