Pass cheerio selectors as a variable

I couldn’t get any responses on other forums so I’ll try here although this is technically not a bug report. Is it possible to do something like this with cheerio?

This is a simplified example. I need to somehow store cheerio selectors in a variable or object and then call request function repeatedly with different set of selectors.

var request = require('request');
var cheerio = require('cheerio');


// None of the below approaches work
// var person = "$('li.person')";  <-- this one is passed as a string
// var person = $('li.person');  <-- this one gives error because $ is not defined

request( 'http://example.com', function(error, response, body) {
  $ = cheerio.load(body);
  var $people = person;

  $people.each(function() {
    console.log('Person: ' + $(this).text());
  }
});

[jugglinmike]

@finspin You might be able to do something quick-and-dirty with eval, but I wouldn’t recommend it. Instead, I can think of two options:

1. Define a more expressive data structure that you can “map” to selections when you’re ready to scrape the page, i.e.

var sites = {
  site1: {
    name: "Site 1",
    url: "http://example1.com",
      selector: ["li.person", "a"]
  }
};

…this is nice because it’s declarative (so you can easily save it to a database), but it will also take some extra thought: you’ll have to design an expressive declarative form for traversing documents, and you’ll have to intelligently iterate through it when the time comes to scrape.

2. Define a function that accepts the “loaded” Cheerio object and makes the selection as necessary:

var sites = {
  site1: {
    name: "Site 1",
    url: "http://example1.com",
    scrape: function($) {
      $('li.person').each(function() {
        console.log($(this).find('a').text());
      });
    }
  }
};

// later...
      function(error, response, body) {
        var $ = cheerio.load(body);
        company.scrape($);
      }

[jugglinmike]

I’m glad it’s working out for you!