Loading...
 
Skip to main content
16
Page actions
PDF Print Get a short URL Open in WebDAV

Elasticsearch

See also OpenSearch and Manticore Search

"Elasticsearch is a distributed, RESTful, free/open source search server based on Apache Lucene." Elasticsearch brings faster search, better search and some Natural Language Processing features.

It is interoperable with Tiki, starting in Tiki12.

1.1. Features enabled by Elasticsearch

1.2. How to configure

Configurable from tiki-admin.php?page=search
Configurable from tiki-admin.php?page=search

1.3. One server vs a cluster

Elasticsearch can be installed as a cluster of servers but can also run on the same server as Tiki. If you have a very high volume of content and have performance issues, you can install on a separate server, set up a cluster or get it as SaaS. You should not install Elasticsearch on a server that has a small amount of RAM (ex.: 1 gig)

1.4. Support matrix

.x means the latest release of that branch

Tiki Elasticsearch
24.4+, 25.2+ 2.x, 5.x, 6.x, 7.x and 8.x
21.x, 22.x, 23.x, 24.x and 25.x 2.x, 5.x, 6.x and 7.x
19.x and 20.x 2.x, 5.x and 6.x
18.x 2.x and 5.x (Sites with a lot of data require a fix from 2018-04-05 which requires 18.2 and Elasticsearch 5.x)
15.x 1.x and 2.x
12.x 1.x and 2.x

Upgrading to Elasticsearch 8

  • When upgrading from pre 7.17 version: You may need to hard-deleted the whole Elasticsearch folder and rebuild index.
  • 8.x comes with security module enabled by default which asks for Basic auth every REST API request: these can be set in The Tiki admin panel. If you know what you are doing, and have other security (ex.: Elasticsearch on same server as Tiki, protected by firewall, etc.), you can turn off security:
    Copy to clipboard
    xpack.security.enabled: false
    in elasticsearch.yml config file.

Upgrading to Elasticsearch 5

Since Elasticsearch automatically updates 2.x index to 5.x first time it starts - you can see some errors then. First time it rebuilds index might also show errors due to the automatic upgrade problems. However, once it builds the index on 5.x, then further rebuilds must not have errors. So, 2 index rebuilds after upgrade is the safest path to go.

1.5. Factoids

1.6. Installation

1.7. Peculiarities

See: PluginList filter control block

1.8. GUI

1.9. Kibana

Kibana: https://www.elastic.co/products/kibana. Some installation notes:
I mostly followed these instructions but haven't got Logstash working yet, and I already had Apache installed so didn't use Nginx, but added this extra apache config file: /usr/local/apache/conf/includes/pre_virtualhost_global.conf

Copy to clipboard
ProxyRequests On

ProxyPass /app/kibana http://127.0.0.1:5601/app/kibana
ProxyPassReverse /app/kibana http://127.0.0.1:5601/app/kibana

ProxyPass /bundles http://127.0.0.1:5601/bundles
ProxyPassReverse /bundles http://127.0.0.1:5601/bundles

ProxyPass /elasticsearch http://127.0.0.1:9200/
ProxyPassReverse /elasticsearch http://127.0.0.1:9200/

# One of these blocks is probably redundant, and you need to make the htpasswd.users file yourself
<Directory /opt/kibana>
    Order deny,allow
    Allow from all
    AuthType Basic
    AuthName "Restricted Area"
    AuthUserFile /opt/kibana/config/htpasswd.users
    Require valid-user
</Directory>

<Proxy *>
    Order deny,allow
    Allow from all
    AuthType Basic
    AuthName "Restricted Area"
    AuthUserFile /opt/kibana/config/htpasswd.users
    Require valid-user
</Proxy>
Note: You need to get the right version of Kibana to match your version of Elastic, I am currently on Elastic 2.4 with Kibana 4.6

1.10. SaaS options


The official Elasticsearch Service from the creators: site link

Version:

  • v.5x, v.6x (see not below)


Cost:

  • roughly $30/GB RAM

  • 1GB RAM / 24GB disk space = $50/month.
  • 4GB RAM / 96GB disk space = $121/month.
  • 64GB RAM / 1.5TB disk space = $1766/month
  • 256GB RAM / 6TB disk space = $6191/month.


Kibana?

  • yes (ELK stack)


Other Notes:

  • "We make the two latest minor versions of the latest major version available for provisioning. In addition, the latest minor version of the previous major version is also available."


Amazon AWS site link

Version:

  • v.1x - v.6x


Cost:

  • roughly $13/GB RAM
    • ‘General Purpose’
    • disc. space = $0.135/GB disk space

  • 1GB RAM / 20GB disk space =$16/month
  • 4GB RAM / 100GB disk space = $67/month
  • 64GB RAM / 1TB disk space =$1004/month


Kibana?

  • yes (ELK stack)


Other Notes:

Bonsai site link

Version:

  • non-dedicated: 5x, 6x,
  • dedicated: region dependent, from v.1x- v.6x


Cost:
SHARED (3 datacenters):

  • roughly: $50/GB (RAM.)

  • Free: 125MB RAM / 125MB disk space
  • 500MB RAM / 2GB disk space = $20/month.
  • 1.25GB RAM / 5GB disk space = $50/month.
  • 3GB RAM / 30GB disk space = $150/month.
  • 12GB RAM / 120GB disk space = $600/month.


DEDICATED:

  • $60/GB (RAM.)

  • 10Gb / 80GB disk space = $600/month.
  • 20GB RAM / 200GB disk space = $1200/month.
  • 40GB RAM / 400GB disk space = $2400/month.


Kibana?

  • yes (ELK stack)


Other Notes:

Scalefastr site link

Version:

  • v.1x - v.5x


Cost:
(no small packages)

  • roughly $8/GB (RAM)

  • 256GB RAM / 4TB (4node) = $2045/month.
  • 384GB RAM / 6TB (6node) = $2943/month.
  • 512GB RAM / 8TB (6node) = $4090/month.


Kibana?

  • yes (ELK stack)


Other Notes:

  • “We only offer storage solutions with a minimum of four nodes. This allows for three total copies of data plus one server to handle failures. Since we deploy on bare metal, this means we're a bit monthre expensive for smaller clusters but they are also much faster than our competitors. We're also much much monthre affordable on the high end. Usually 30-50% less than our competitors for data at large scale."


QBox site link

Version:

  • v.1x - v.5x


Cost:

  • $12 - 40/GB RAM

  • 1GB RAM / 20GB disk space = $40/month
  • 4GB RAM / 80GB disk space = $90/month
  • 64GB RAM / 1.2TB disk space = $1342/month
  • 768GB RAM / 15.3TB disk space = $10,043/month.


Kibana?

  • yes (ELK stack)


Other Notes:

Searchly site link

Version:

  • v.1x - v.5x


Cost:
(only shared RAM, see below)

  • 3 ‘Indeces’ / 1GB disk space = $9/month.
  • 9 ‘Indeces’ / 6GB disk space = $59/month.
  • 18 ‘Indeces’ / 30GB disk space = $199/month.
  • 44 ‘Indeces’ / 100GB disk space = $799/month.


Kibana?

  • yes (ELK stack)


Other Notes:

  • "Indices count do not influence the search performance. Searchly is using shared architecture, therefore there are no dedicated ram."

Logz site link

Version:

  • v.5x


Cost:

  • roughly: $46 - 57 /GB disc
    • see note below about RAM

  • Free 3GB (disc, not RAM.);
  • 5GB disk space = $290/month.
  • 100GB disk space = $4620/month.
  • 100GB - 15TB disk space = ‘contact us’


Kibana?

  • yes (ELK stack)


Other Notes:

  • "We are not priced based on RAM allocation because of multi-tenancy and a couple other factors-- instead we charge based on daily log volume ingested and retention time (e.g. 5 GB/day, 14 days retention)."
  • public API's on Github
  • free trial on site


ObjectRocket site link

Version:

  • v.5x - v.6x


Cost:

  • roughly $30/GB RAM

  • 2GB RAM / 16GB (2 x 8GB nodes) = $60/month.
  • 4GB RAM / 32GB (2 x 16GB nodes) = $240/month.
  • 12GB RAM / 96GB (3 x 32GB nodes) = $1,379/month.
  • 64GB RAM / 512GB (2 x 256GB nodes) = $5517/month



Kibana?

  • yes (ELK stack)


Other Notes:

  • price changed - updated Object Rocket May 20, 2018

  • as of April 2018

1.11. Related


alias

History Source
Slideshow