Laravel uses Elasticsearch

Elasticsearch

Elasticsearch is an open source search engine based on Apache Lucene (TM), Lucene can be considered as the most advanced, best performing and most functional search engine library to date, both in open source and proprietary fields. However, Lucene is just a library. To use its power, you need to use Java and integrate it into your application. Lucene is very complex, and you need a deep understanding of retrieval to understand how it works.

Elasticsearch is also written in Java and uses Lucene for indexing and search functionality, but its purpose is to make full-text search simple and hide the complexities of Lucene through a simple and coherent RESTful API.
However, Elasticsearch is more than just Lucene and a full-text search engine, it also provides:

Distributed real-time file storage, every field is indexed and searchable
Distributed search engine for real-time analysis
Scale to hundreds of servers to process petabytes of structured or unstructured data
And, all of these features are integrated into a single server that your application can interact with through simple RESTful APIs, clients in various languages, and even the command line. Getting started with Elasticsearch is very simple, it provides many reasonable defaults, and hides complex search engine theory from beginners. It works out of the box (installation ready to use) and can be used in a production environment with minimal learning.

Elasticsearch is licensed under the Apache 2 License and is free to download, use, and modify.

ElasticSearch installation

ElasticSearch is already integrated in Laradock. We can directly use:

# start up
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --name=elasticsearch docker.elastic.co/elasticsearch/elasticsearch:7.11.1

# If you need to install the plugin, execute the command:
docker exec -it elasticsearch /bin/bash

/usr/share/elasticsearch/bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.11.1/elasticsearch-analysis-ik-7.11.1.zip

# restart the container
docker restart elasticsearch

Note: The vm.max_map_count kernel setting must be set to at least 262144 for production use.

Since I am in a centos 7 environment, I set it directly in the system settings:

sysctl -w vm.max_map_count=262144

Default username and password: "elastic", "changeme", port number: 9200

ElasticHQ

ElasticHQ is an open source application that offers a simplified interface for managing and monitoring Elasticsearch clusters.
Management and Monitoring for Elasticsearch.
http://www.elastichq.org/

Real-Time Monitoring
Full Cluster Management
Full Cluster Monitoring
Elasticsearch Version Agnostic
Easy Install - Always On
Works with X-Pack

Enter our Elasticsearch Host to enter the background.

By default it creates:

A cluster Cluster: laradock-cluster
A node Node: laradock-node
An index Index: .elastichq

IK tokenizer installation

ElasticSearch is mainly used for the search of your own blog or official account articles, so you need to choose a Chinese tokenizer to use together. Here we just recommend using the IK tokenizer. Let's start to install the plugin that corresponds to the ElasticSearch version (7.5.1):
Elasticsearch-analysis-ik

// install plugin
docker exec -it elasticsearch /bin/bash

/usr/share/elasticsearch/bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.11.1/elasticsearch-analysis-ik-7.11.1.zip

Note: You can download the zip file first, and then install it, the speed will be faster.

Check word segmentation effect

According to the Elasticsearch API test, the effect of word segmentation has reached:

curl -X POST "http://your_host/_analyze?pretty" -H 'Content-Type: application/json' -d'
{
  "analyzer": "ik_max_word",
  "text": "I am Chinese"
}
'
{
  "tokens" : [
    {
      "token" : "I",
      "start_offset" : 0,
      "end_offset" : 1,
      "type" : "CN_CHAR",
      "position" : 0
    },
    {
      "token" : "Yes",
      "start_offset" : 1,
      "end_offset" : 2,
      "type" : "CN_CHAR",
      "position" : 1
    },
    {
      "token" : "Chinese",
      "start_offset" : 2,
      "end_offset" : 5,
      "type" : "CN_WORD",
      "position" : 2
    },
    {
      "token" : "China",
      "start_offset" : 2,
      "end_offset" : 4,
      "type" : "CN_WORD",
      "position" : 3
    },
    {
      "token" : "Chinese",
      "start_offset" : 3,
      "end_offset" : 5,
      "type" : "CN_WORD",
      "position" : 4
    }
  ]
}

Combining with Laravel

Although Elasticsearch officially provides a plugin for the corresponding PHP version, we still want to integrate it more closely with Laravel, so here we choose to use it in conjunction with Scout, using the tamayo/laravel-scout-elastic plugin.

composer require tamayo/laravel-scout-elastic

composer require laravel/scout
# Publish Scout's configuration file
php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"

#Modify the driver to elasticsearch:
# config/scout.php
'driver' => env('SCOUT_DRIVER', 'elasticsearch')

create index

There are several ways to create an index, one of which can be created directly using the Ela visualization tool ElasticHQ.

Next we need to update this index, supplement the Mappings section, you can use Postman.

Another way is to use the Artisan command line functionality that comes with Laravel.

Here we recommend using the Artisan command line.

php artisan make:command ESOpenCommand

According to the official website prompt, we can send a PUT request to the Elasticsearch server on ESOpenCommand. Here, with the help of the PHP plugin provided by Elasticsearch, when we use the tamayo/laravel-scout-elastic plugin, the Elasticsearch PHP plugin has been installed:

Now we can use the plug-in to create our Index and look at the code directly:

public function handle()
    {
    $host = config('scout.elasticsearch.hosts');
    $index = config('scout.elasticsearch.index');
    $client = ClientBuilder::create()->setHosts($host)->build();

    if ($client->indices()->exists(['index' => $index])) {
        $this->warn("Index {$index} exists, deleting...");
        $client->indices()->delete(['index' => $index]);
    }

    $this->info("Creating index: {$index}");

    return $client->indices()->create([
        'index' => $index,
        'body' => [
            'settings' => [
                'number_of_shards' => 1,
                'number_of_replicas' => 0
            ],
            'mappings' => [
                '_source' => [
                    'enabled' => true
                ],
                'properties' => [
                    'id' => [
                        'type' => 'long'
                    ],
                    'title' => [
                        'type' => 'text','analyzer' => 'ik_max_word',
                        'search_analyzer' => 'ik_smart'
                    ],
                    'subtitle' => [
                        'type' => 'text',
                        'analyzer' => 'ik_max_word',
                        'search_analyzer' => 'ik_smart'
                    ],
                    'content' => [
                        'type' => 'text',
                        'analyzer' => 'ik_max_word',
                        'search_analyzer' => 'ik_smart'
                    ]
                ],
            ]
        ]
    ]);
}

Well, we execute Kibana and see that we have created the Index:

Note Kibana local Docker installation:

docker pull docker.elastic.co/kibana/kibana:7.11.1
docker run -d --name kibana --link elasticsearch -e ELASTICSEARCH_HOSTS=http://elasticsearch:9200 -p 5601:5601 docker.elastic.co/kibana/kibana:7.11.1

In order to verify whether Index is available, you can insert a piece of data to see:

curl -XPOST your_host/coding01_open/_create/1 -H 'Content-Type:application/json' -d '{"content":"Investigation of conflict between Chinese and Korean fishing police: Korean police detain an average of 1 Chinese fishing boat per day"}

You can view the corresponding data through the browser:

With Index, the next step is to combine Laravel to import, update, query and other operations.

Laravel Model usage

The Laravel framework has recommended the use of Scout full-text search for us. We only need to add the official content to the Article Model. It is very simple. It is recommended that you read the Scout documentation: [Scout full-text search "Laravel 6 Chinese Documentation"](https ://learnku.com/docs/laravel/6.x/scout/5191 "Scout full-text search "Laravel 6 Chinese Documentation""), the code is directly below:

<?php

namespace App;

use App\Tools\Markdowner;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\SoftDeletes;
use Laravel\Scout\Searchable;

class Article extends Model
{
    use Searchable;

    protected $connection = 'blog';
    protected $table = 'articles';
    use SoftDeletes;

    /**
     * The attributes that should be mutated to dates.
     *
     * @var array
     */
    protected $dates = ['published_at', 'created_at', 'deleted_at'];

    /**
     * The attributes that are mass assignable.
     *
     * @var array
     */
    protected $fillable = [
        'user_id',
        'last_user_id',
        'category_id',
        'title',
        'subtitle',
        'slug',
        'page_image',
        'content',
        'meta_description',
        'is_draft',
        'is_original',
        'published_at',
        'wechat_url',
    ];

    protected $casts = [
        'content' => 'array'
    ];

    /**
     * Set the content attribute.
     *
     * @param $value
     */
    public function setContentAttribute($value)
    {
        $data = [
            'raw' => $value,
            'html' => (new Markdowner)->convertMarkdownToHtml($value)
        ];

        $this->attributes['content'] = json_encode($data);
    }

    /**
     * Get searchable data for the model
     *
     * [[@return](https://learnku.com/users/31554)](https://learnku.com/users/31554) array
     */
    public function toSearchableArray()
    {
        $data = [
            'id' => $this->id,
            'title' => $this->title,
            'subtitle' => $this->subtitle,
            'content' => $this->content['html']
        ];

        return $data;
    }

    public function searchableAs()
    {
        return '_doc';
    }
}

Scout provides the Artisan command import to import all existing records into the search index.

php artisan scout:import "App\Article"

Looking at Kibana, 12 records have been stored, which matches the number of records in the database.

With the data, we can test to see if we can query the data.

Still the same, create a command:

class ElasearchCommand extends Command
{
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'command:search {query}';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'Command description';

    /**
     * Create a new command instance.
     *
     * [[@return](https://learnku.com/users/31554)](https://learnku.com/users/31554) void
     */
    public function __construct()
    {
        parent::__construct();
    }

    /**
     * Execute the console command.
     *
     * [[@return](https://learnku.com/users/31554)](https://learnku.com/users/31554) mixed
     */
    public function handle()
    {
        $article = Article::search($this->argument('query'))->first();
        $this->info($article->title);
    }
}

This is my titles, I just type in a keyword: "list" to see if I can find it.

Summarize

Completed as a whole:

• Elasticsearch installation;
• Elasticsearch IK tokenizer plugin installation;
• Installation and simple use of Elasticsearch visualization tools ElasticHQ and Kibana;
• use of Scout;
• Use Elasticsearch with Scout.