How implement Elasticsearch in cget

Hi @Dunglas,

I’ve read somewhere that you use frequently Elasticsearch with api-platform. I tried to serialize my $finder->findPaginated($query) result as the same format as the cget result (in jsonld).

[EDIT] I don’t want to just returns IDs because I want to send my aggregations to the front too. I want to do this way because it let me build my search dynamically in front with the result of the cget).

I don’t know how to implement my searcher services in cget action and disable the filters. I just want to give my result for serializing.

Might you please provide an example of the way you do to implement ES in your ApiPlatform projects. I don’t understand if I should make a customFilter (which seem to only provide a query builder to api platform) that completely overrides the filter feature provided with ApiPlatform.

Thanks a lot,

Ksom

[kbyjoel]

Unfortunately, it’s not possible to add a “buckets” column in hydra mapping via a filter because the ApiPlatform\Core\Hydra\Serializer\CollectionFiltersNormalizer is restrictive (at least on api-platform version from 2.1 to 2.3) :

    /**
     * Returns the content of the Hydra search property.
     *
     * @param FilterInterface[] $filters
     */
    private function getSearch(string $resourceClass, array $parts, array $filters): array
    {
        $variables = [];
        $mapping = [];
        foreach ($filters as $filter) {
            foreach ($filter->getDescription($resourceClass) as $variable => $data) {
                $variables[] = $variable;
                $mapping[] = [
                    '@type' => 'IriTemplateMapping',
                    'variable' => $variable,
                    'property' => $data['property'],
                    'required' => $data['required'],
                ];
            }
        }

        return [
            '@type' => 'hydra:IriTemplate',
            'hydra:template' => sprintf('%s{?%s}', $parts['path'], implode(',', $variables)),
            'hydra:variableRepresentation' => 'BasicRepresentation',
            'hydra:mapping' => $mapping,
        ];
    }

Currently, the only way to achieve the aggregation normalization, is to put this information in the property column of the description of your custom filter. Perhaps it would be fine to authorize the extension of hydra mapping information via CustomFilters to make work what @ksom wrote last year.

The method getSearch of ApiPlatform\Core\Hydra\Serializer\CollectionFiltersNormalizer could be something like this:

    /**
     * Returns the content of the Hydra search property.
     *
     * @param FilterInterface[] $filters
     */
    private function getSearch(string $resourceClass, array $parts, array $filters): array
    {
        $variables = [];
        $mapping = [];
        foreach ($filters as $filter) {
            foreach ($filter->getDescription($resourceClass) as $variable => $data) {
                $variables[] = $variable;
                $propertyMapping = ['@type' => 'IriTemplateMapping', 'variable' => $variable];
                foreach ($data as $additionnalKey => $additionnalValue) {
                    if ($additionnalKey != '@type' && $additionnalKey != 'variable') {
                        $propertyMapping[$additionnalKey] = $additionnalValue;
                    }
                }
                $mapping[] = $propertyMapping;
            }
        }

        return [
            '@type' => 'hydra:IriTemplate',
            'hydra:template' => sprintf('%s{?%s}', $parts['path'], implode(',', $variables)),
            'hydra:variableRepresentation' => 'BasicRepresentation',
            'hydra:mapping' => $mapping,
        ];
    }

What do you think about it ?

[ksom]

Hi @Simperfit,

I have one SearcherService:

class SearcherService
{
    protected $cachedResults = [];
    protected $cachedAggregations = [];

    public function search($entityName, $refresh = true)  
    {
       // Use entityName to know what index you want to query
       
       // Used if you want to do a new search
       if ($refresh) {
            $this->clearCache();
        }

        // build your search
        $result = $finder->findPaginated($query)
            ->setMaxPerPage(isset($params['_per_page']) ? $params['_per_page'] : 10)
            ->setCurrentPage(isset($params['_page']) ? $params['_page'] : 1);

        $aggregations = $result->getAdapter()->getAggregations();
        // Used to add a ['label']['en'] and ['label']['fr'] to provide a label to print in the front application for the filter value
        foreach ($aggregations as &$aggregation) {
            $this->postProcessAggregation($aggregation);
        }
        

        $currentPageResult = $result->getCurrentPageResults();
        $this->cachedResults = $currentPageResult;
        $this->cachedAggregations = $aggregations;
    }


    public function clearCache()
    {
        $this->cachedResults = [];
        $this->cachedAggregations = [];
    }

    public function hasCachedResult()
    {
        return count($this->cachedResults) > 0;
    }


    public function getResults()
    {
        return count($this->cachedResults) > 0 ? $this->cachedResults : [];
    }

    public function getAggregations()
    {
        return count($this->cachedResults) > 0 ? $this->cachedAggregations : [];
    }
}

I have one custom provider ElasticsearchCollectionDataProvider:

<?php

namespace AppBundle\DataProvider;

use ApiPlatform\Core\DataProvider\CollectionDataProviderInterface;
use ApiPlatform\Core\DataProvider\ItemDataProviderInterface;
use ApiPlatform\Core\Exception\ResourceClassNotSupportedException;
use AppBundle\Service\SearcherService;
use Doctrine\ORM\EntityManager;
use Symfony\Component\HttpFoundation\RequestStack;
use Symfony\Component\Security\Core\Authentication\Token\Storage\TokenStorage;

final class ElasticsearchCollectionDataProvider implements CollectionDataProviderInterface
{
    /**
     * @var SearcherService
     */
    protected $searcher;

    /**
     * @var RequestStack
     */
    protected $requestStack;

    /**
     * @param SearcherService $searcher
     * @param RequestStack $requestStack
     */
    public function __construct(SearcherService $searcher, RequestStack $requestStack)
    {
        $this->searcher = $searcher;
        $this->requestStack = $requestStack;
    }


    public function getCollection(string $resourceClass, string $operationName = null)
    {
        // Add here all entities that's you want to query with ElasticSearch
        if (!in_array($resourceClass, [Post::class, Agency::class, Organization::class])) {
            throw new ResourceClassNotSupportedException();
        }

        $index = explode('\\', $resourceClass);
        $index = strtolower(end($index));

        $this->searcher->search($index);

        return $this->searcher->getResults();
    }
}

I have an ElasticsearchFilter:

<?php

namespace AppBundle\Filter;

use ApiPlatform\Core\Api\FilterInterface;
use ApiPlatform\Core\Bridge\Doctrine\Orm\Filter\AbstractFilter;
use ApiPlatform\Core\Bridge\Doctrine\Orm\Util\QueryNameGeneratorInterface;
use AppBundle\Service\SearcherService;
use Doctrine\Common\Persistence\ManagerRegistry;
use Elastica\Multi\Search;
use Symfony\Component\Config\Resource\ResourceInterface;
use Symfony\Component\HttpFoundation\Request;
use Doctrine\ORM\QueryBuilder;
use Symfony\Component\HttpFoundation\RequestStack;

class ElasticsearchFilter extends AbstractFilter
{
    /** @var SearcherService */
    private $engine;

    /**
     * @param ManagerRegistry $managerRegistry
     * @param RequestStack $requestStack
     * @param SearcherService $engine
     */

    public function __construct(ManagerRegistry $managerRegistry, RequestStack $requestStack, SearcherService $engine)
    {
        parent::__construct($managerRegistry, $requestStack);
        $this->engine = $engine;
    }

    public function getDescription(string $resourceClass) : array
    {
        // I override the description to add a buckets array key to put my aggregations
        $description = [];
        foreach ($this->engine->getAggregations() as $property => $aggregation) {
            $description[$property] = [
                "property" => $aggregation['property'],
                "required" => false,
                'buckets' => $aggregation['buckets'],
            ];
        }

        return $description;
    }

    /**
     * Passes a property through the filter.
     *
     * @param string $property
     * @param mixed $value
     * @param QueryBuilder $queryBuilder
     * @param QueryNameGeneratorInterface $queryNameGenerator
     * @param string $resourceClass
     * @param string|null $operationName
     */
    protected function filterProperty(string $property, $value, QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, string $operationName = null)
    {}
}

To resume: I have a searcher, a custom filter and a custom provider. The searcher save results and aggregations in cached properties. I call the SearcherService#search() method in a custom Provider (because providers are called before filters) and I return the SearcherService#getResults method. In the custom Filter I override getDescription method to foreach over SearcherService#getAggregations() to add a buckets key in the description for each aggregation.

I can’t give you a complete example at this moment. I’m sorry to answer you just now but I’m really busy actually.

If you have any questions related to this post please let me know I will do my best to answer you quickly.

Hope this can help!