question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

why oplog.timestamp is null?

See original GitHub issue

when I execute mongo-connector 2.4 sync to elasticsearch2.3.4 data sync correctly ,no errors,but oplog.timestamp file has nothing. does not it update timestamp like [“rs0”, 6339713900350538292] ? how and when mongo-connector will update oplog.timestamp? when I shutdown mongo-connector,and want to continue syn,I will wait too long ,data do not sync into elasticsearch . thanks,urgent!

I run mongo-connector like “mongo-connector -c config.json” and my config.json like here:

{
  "mainAddress": "*.*.*.*:27017",
  "oplogFile": "/data/opt/mongo-connector/oplog.timestamp",
  "verbosity": 1,
  "logging": {
    "type": "file",
    "filename": "/data/opt/mongo-connector/mongo-connector.log",
    "__rotationWhen": "D",
    "__rotationInterval": 1,
    "__rotationBackups": 10
  },
  "fields": ["_id",,"label","pinyin","begin_time","end_time"],
  "namespaces": {
    "include": ["aa.a", "bb.b"],
    "mapping": {
      "aa.a": "aa.a",
      "bb.b": "bb.b"
    }
  },
  "docManagers": [
    {
      "docManager": "elastic2_doc_manager",
      "targetURL": "*.*.*.*:9200",
      "bulkSize": 4000,
      "autoCommitInterval": null,
      "args": {
        "clientOptions": {"timeout": 300}
      }
    }
  ]
}

Edit: formatted json

Issue Analytics

  • State:open
  • Created 7 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
ShaneHarveycommented, Oct 14, 2016

We should add a --version flag in the future but for now you can run:

$ pip freeze
elastic2-doc-manager==0.2.0
elasticsearch==2.4.0
mongo-connector==2.4.1
pymongo==3.3.0
pysolr==3.5.0
requests==2.11.1
urllib3==1.18
1reaction
ShaneHarveycommented, Oct 13, 2016

Are you using mongo-connector v2.4 or v2.4.1?

The connector should update the timestamp to the latest replicated oplog entry every second. If the checkpoint is never set the cause may be different between versions. Can you post the log file or re-run with a higher logging "verbosity": 3?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Cannot use timestamp from oplog for filtering - Stack Overflow
MongoDB is finicky about field names that begin with dollar sign. You an use an extra stage to convert that object to an...
Read more >
[SERVER-57585] oplog_visibility.js prints null 'next' timestamp ...
It seems the NextObserved timestamp is always null even when the current index is not the at end of the list.
Read more >
Replica Set Oplog — MongoDB Manual
oplog entries are time-stamped. The oplog window is the time difference between the newest and the oldest timestamps in the oplog . If...
Read more >
error while getting last oplog entry for begin timestamp: find ...
You cannot add a MongoDB 3.4.x member to a MongoDB 3.0.x replica set. Replication between mixed versions is only supported for adjacent ...
Read more >
MongoDB Oplog - StreamSets Documentation
When you start the pipeline, the MongoDB Oplog origin starts reading from the beginning of the Oplog by default. You can configure a...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found