History: Elasticsearch
Preview of version: 78
Elasticsearch
See also OpenSearch and Manticore Search
"Elasticsearch is a distributed, RESTful, free/open source search server based on Apache Lucene." Elasticsearch brings faster search, better search and some Natural Language Processing features.
It is interoperable with Tiki, starting in Tiki12.
1.1. Features enabled by Elasticsearch
- Faceted search, which also be used through PluginCustomSearch
- Stored Search
- Natural Language Processing
- Module More Like This
- Federated Search
1.2. How to configure
1.3. One server vs a cluster
Elasticsearch can be installed as a cluster of servers but can also run on the same server as Tiki. If you have a very high volume of content and have performance issues, you can install on a separate server, set up a cluster or get it as SaaS. You should not install Elasticsearch on a server that has a small amount of RAM (ex.: 1 gig)
1.4. Support matrix
.x means the latest release of that branch
| Tiki | Elasticsearch |
| 21.9+, 24.4+, 25.2+ | 2.x, 5.x, 6.x, 7.x and 8.x |
| 21.x, 22.x, 23.x, 24.x and 25.x | 2.x, 5.x, 6.x and 7.x |
| 19.x and 20.x | 2.x, 5.x and 6.x |
| 18.x | 2.x and 5.x (Sites with a lot of data require a fix from 2018-04-05 which requires 18.2 and Elasticsearch 5.x) |
| 15.x | 1.x and 2.x |
| 12.x | 1.x and 2.x |
Upgrading to Elasticsearch 8
- When upgrading from pre 7.17 version: You may need to hard-deleted the whole Elasticsearch folder and rebuild index.
- 8.x comes with security module enabled by default which asks for Basic auth every REST API request: these can be set in The Tiki admin panel. If you know what you are doing, and have other security (ex.: Elasticsearch on same server as Tiki, protected by firewall, etc.), you can turn off security: in elasticsearch.yml config file.Copy to clipboardxpack.security.enabled: false
Upgrading to Elasticsearch 5
Since Elasticsearch automatically updates 2.x index to 5.x first time it starts - you can see some errors then. First time it rebuilds index might also show errors due to the automatic upgrade problems. However, once it builds the index on 5.x, then further rebuilds must not have errors. So, 2 index rebuilds after upgrade is the safest path to go.
1.5. Factoids
1.6. Video
[+]1.7. Installation
- https://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html
- http://wikisuite.org/How-to-install-Elasticsearch-on-ClearOS
1.8. Peculiarities
See: PluginList filter control block
1.9. GUI
- http://www.adminer.org/ supports Elasticsearch
- Kibana
- Plotly
1.10. Kibana
Kibana: https://www.elastic.co/products/kibana. Some installation notes:
I mostly followed these instructions but haven't got Logstash working yet, and I already had Apache installed so didn't use Nginx, but added this extra apache config file: /usr/local/apache/conf/includes/pre_virtualhost_global.conf
ProxyRequests On ProxyPass /app/kibana http://127.0.0.1:5601/app/kibana ProxyPassReverse /app/kibana http://127.0.0.1:5601/app/kibana ProxyPass /bundles http://127.0.0.1:5601/bundles ProxyPassReverse /bundles http://127.0.0.1:5601/bundles ProxyPass /elasticsearch http://127.0.0.1:9200/ ProxyPassReverse /elasticsearch http://127.0.0.1:9200/ # One of these blocks is probably redundant, and you need to make the htpasswd.users file yourself <Directory /opt/kibana> Order deny,allow Allow from all AuthType Basic AuthName "Restricted Area" AuthUserFile /opt/kibana/config/htpasswd.users Require valid-user </Directory> <Proxy *> Order deny,allow Allow from all AuthType Basic AuthName "Restricted Area" AuthUserFile /opt/kibana/config/htpasswd.users Require valid-user </Proxy>
A newer simpler way is to use: http://wikisuite.org/How-to-install-Kibana-on-ClearOS
1.11. SaaS options
The official Elasticsearch Service from the creators: site link
Version:
- v.5x, v.6x (see not below)
Cost:
- roughly $30/GB RAM
- 1GB RAM / 24GB disk space = $50/month.
- 4GB RAM / 96GB disk space = $121/month.
- 64GB RAM / 1.5TB disk space = $1766/month
- 256GB RAM / 6TB disk space = $6191/month.
Kibana?
- yes (ELK stack)
Other Notes:
- "We make the two latest minor versions of the latest major version available for provisioning. In addition, the latest minor version of the previous major version is also available."
Amazon AWS site link
Version:
- v.1x - v.6x
Cost:
- roughly $13/GB RAM
- ‘General Purpose’
- disc. space = $0.135/GB disk space
- 1GB RAM / 20GB disk space =$16/month
- 4GB RAM / 100GB disk space = $67/month
- 64GB RAM / 1TB disk space =$1004/month
Kibana?
- yes (ELK stack)
Other Notes:
Bonsai site link
Version:
- non-dedicated: 5x, 6x,
- dedicated: region dependent, from v.1x- v.6x
Cost:
SHARED (3 datacenters):
- roughly: $50/GB (RAM.)
- Free: 125MB RAM / 125MB disk space
- 500MB RAM / 2GB disk space = $20/month.
- 1.25GB RAM / 5GB disk space = $50/month.
- 3GB RAM / 30GB disk space = $150/month.
- 12GB RAM / 120GB disk space = $600/month.
DEDICATED:
- $60/GB (RAM.)
- 10Gb / 80GB disk space = $600/month.
- 20GB RAM / 200GB disk space = $1200/month.
- 40GB RAM / 400GB disk space = $2400/month.
Kibana?
- yes (ELK stack)
Other Notes:
Scalefastr site link
Version:
- v.1x - v.5x
Cost:
(no small packages)
- roughly $8/GB (RAM)
- 256GB RAM / 4TB (4node) = $2045/month.
- 384GB RAM / 6TB (6node) = $2943/month.
- 512GB RAM / 8TB (6node) = $4090/month.
Kibana?
- yes (ELK stack)
Other Notes:
- “We only offer storage solutions with a minimum of four nodes. This allows for three total copies of data plus one server to handle failures. Since we deploy on bare metal, this means we're a bit monthre expensive for smaller clusters but they are also much faster than our competitors. We're also much much monthre affordable on the high end. Usually 30-50% less than our competitors for data at large scale."
QBox site link
Version:
- v.1x - v.5x
Cost:
- $12 - 40/GB RAM
- 1GB RAM / 20GB disk space = $40/month
- 4GB RAM / 80GB disk space = $90/month
- 64GB RAM / 1.2TB disk space = $1342/month
- 768GB RAM / 15.3TB disk space = $10,043/month.
Kibana?
- yes (ELK stack)
Other Notes:
Searchly site link
Version:
- v.1x - v.5x
Cost:
(only shared RAM, see below)
- 3 ‘Indeces’ / 1GB disk space = $9/month.
- 9 ‘Indeces’ / 6GB disk space = $59/month.
- 18 ‘Indeces’ / 30GB disk space = $199/month.
- 44 ‘Indeces’ / 100GB disk space = $799/month.
Kibana?
- yes (ELK stack)
Other Notes:
- "Indices count do not influence the search performance. Searchly is using shared architecture, therefore there are no dedicated ram."
Logz site link
Version:
- v.5x
Cost:
- roughly: $46 - 57 /GB disc
- see note below about RAM
- Free 3GB (disc, not RAM.);
- 5GB disk space = $290/month.
- 100GB disk space = $4620/month.
- 100GB - 15TB disk space = ‘contact us’
Kibana?
- yes (ELK stack)
Other Notes:
- "We are not priced based on RAM allocation because of multi-tenancy and a couple other factors-- instead we charge based on daily log volume ingested and retention time (e.g. 5 GB/day, 14 days retention)."
- public API's on Github
- free trial on site
ObjectRocket site link
Version:
- v.5x - v.6x
Cost:
- roughly $30/GB RAM
- 2GB RAM / 16GB (2 x 8GB nodes) = $60/month.
- 4GB RAM / 32GB (2 x 16GB nodes) = $240/month.
- 12GB RAM / 96GB (3 x 32GB nodes) = $1,379/month.
- 64GB RAM / 512GB (2 x 256GB nodes) = $5517/month
Kibana?
- yes (ELK stack)
Other Notes:
- price changed - updated Object Rocket May 20, 2018
- as of April 2018
1.12. Related
- How to configure Tiki for Elasticsearch Service
- Recommendation Engine
- Elasticsearch Query builder extension for Google Chrome
- Logging using ELK Stack