question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Query to fetch single record from 10K total keys is very slow (~12 seconds)

See original GitHub issue
import datetime
from typing import Optional

from redis_om import Field, HashModel, Migrator, get_redis_connection

# This Redis instance is tuned for durability.
REDIS_DATA_URL = "redis://localhost:6380"

class Person(HashModel):
    first_name: str = Field(index=True)
    last_name: str = Field(index=True)
    emp_no: int =  Field(index=True)

# set redis connection
Person.Meta.database = get_redis_connection(url=REDIS_DATA_URL,
                                                  decode_responses=True)
# apply migrations
Migrator().run()

for row_number in range(0,10000):
    person = Person(first_name="John" + str(row_number), last_name="Doe", emp_no=row_number)
    
    result = Person.find(Person.emp_no ==row_number).all()
    if (len(result) == 0):
        person.save()

 
    print(person.pk)

# very slow to query a single record (~12 seconds)
Person.find().sort_by('-emp_no').first()

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:9 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
simonprickettcommented, Apr 22, 2022

@msarm fixed in 0.0.25 which is now available.

0reactions
simonprickettcommented, Apr 25, 2022

@msarm I’ve raised #218 for the first question.

Read more comments on GitHub >

github_iconTop Results From Across the Web

32000 records - simple SELECT is slow - SQLServerCentral
All simple SELECT queries are slow in this database, from 1 to 2 min for 30,000 rows. It's not server performance issue.
Read more >
Chapter 4. Query Performance Optimization - O'Reilly
Connection response is still slow compared to the number of rows MySQL can traverse per second internally, though, which is counted in millions...
Read more >
MySQL select 10 random rows from 600K rows fast
I am getting fast queries (around 0.5 seconds) with a slow cpu, selecting 10 random rows in a 400K registers MySQL database non-cached...
Read more >
SQL OFFSET FETCH Feature: Loading Large Volumes of ...
Implementing the OFFSET FETCH feature within SSIS to load a large volume of data in chunks · RowCount (Int32): Stores the total number...
Read more >
Slow mysql Query? Make your Msqyl Fast and faster! - YouTube
Programming with Matt, Optimizing your mysql server, make it fast as hell, super fast, and even more faster than it was for you...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found