Performance issue for selecting with Many-to-Many relationship

Issue type:

[ ] question [x] bug report [ ] feature request [ ] documentation issue

Database system/driver:

[ ] cordova [ ] mongodb [ ] mssql [x] mysql / mariadb [ ] oracle [ ] postgres [ ] sqlite [ ] sqljs [ ] react-native

TypeORM version:

[ ] latest [ ] @next [x] 0.2.5

Steps to reproduce or a small repository showing the problem:

@Entity()
export class Subscriber {
  @PrimaryGeneratedColumn() id: number;
  @Column({
    type: 'varchar'
  })
  email: string;
  @ManyToMany(type => MailingList, mailingList => mailingList.subscribers)
  mailingLists: MailingList[];
  @Column({
    name: 'del_flag'
  })
  delFlag: Boolean;
  @Column() solicited: Boolean;
  @Column() unsubscribed: Boolean;
}

@Entity('mailing_list')
export class MailingList {
  @PrimaryGeneratedColumn() id: number;
  @Column({
    type: 'varchar'
  })
  name: string;
  @ManyToMany(type => Subscriber, subscriber => subscriber.mailingLists)
  @JoinTable({
    name: 'mailing_list_subscriber',
    joinColumn: { name: 'mailing_list_id', referencedColumnName: 'id' },
    inverseJoinColumn: { name: 'subscriber_id', referencedColumnName: 'id' }
  })
  subscribers: Subscriber[];
}

In table Subscriber I have around 100 000 records and in mailing_list_subscriber around 110 000 records. When I try to run query below it takes significant amount of time (> 1min).

const repo = getManager().getRepository(Subscriber);
let subscribers = repo
  .createQueryBuilder('subscriber')
  .leftJoinAndSelect('subscriber.mailingLists', 'mailingList')
  .where('subscriber.del_flag = 0')
  .skip(10)
  .take(10)
  .getManyAndCount(); 

[pleerock]

Okay so as you can see SQL takes lot of time and it works slowly not because of orm but because of your query. Why SQL is slow? Because when you select data and join lot of other data and select that data sql returns everything in rows and multiplies data which results into millions of returned rows.

Solution is to execute separate query for your subscribers, load only them, then execute separate query for mailing lists and load them, then map mailing lists into subscribers manually in your code.

[pleerock]

you can use left joins, just remove “AndSelect” from there, e.g. leftJoinAndSelect( => leftJoin( - same with inner joins and you’ll have all the filtering in there.