Skip to content

Commit 568aa42

Browse files
kimchykimchy
committed
experiment with guide
1 parent a72fda7 commit 568aa42

39 files changed

+1353
-33
lines changed

‎_includes/header.html

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<div id="header">
22
<a href="/">
3-
<div style="padding-top: 70px; padding-left: 50px">
3+
<div style="padding-top: 70px; margin-left: 50px">
44
<div style="float: left">
55
<img src="/images/logo.png" height="170px">
66
</div>
@@ -12,7 +12,7 @@
1212
</div>
1313
</a>
1414
<div class="clear"></div>
15-
<div style="height: 80px; text-align: center; font-size: 34px; padding-top: 10px">
15+
<div style="height: 80px; font-size: 34px; padding-top: 30px;">
1616
{% if page.title_in_header != false %}
1717
{{ page.title }}
1818
{% endif %}

‎_includes/top_left_menu.html

+6-6
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
<div id="top_left_menu" style="position: absolute; top: 0; left: 0; border: 0;">
2-
<div class="m_link"><a href="/">Home</a></div>
3-
<div class="m_link"><a href="/download">Download</a></div>
4-
<div class="m_link"><a href="/community">Community</a></div>
5-
<div class="m_link"><a href="/blog">Blog</a></div>
6-
<div class="m_link"><a href="/videos">Videos</a></div>
7-
<div class="m_link"><a href="/docs">Docs</a></div>
2+
<div class="m_link"><a href="/">home</a></div>
3+
<div class="m_link"><a href="/download">download</a></div>
4+
<div class="m_link"><a href="/community">community</a></div>
5+
<div class="m_link"><a href="/blog">blog</a></div>
6+
<div class="m_link"><a href="/videos">videos</a></div>
7+
<div class="m_link"><a href="/docs">docs</a></div>
88
</div>

‎_layouts/blogpost.html

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
<a href="http://twitter.com/share" class="twitter-share-button" data-url="http://www.elasticsearch.org{{page.url}}" data-text="{{page.title}}" data-count="horizontal" data-via="elasticsearch" data-related="kimchy:Shay Banon: Founder of elasticsearch">Tweet</a><script type="text/javascript" src="http://platform.twitter.com/widgets.js"></script>
77
</div>
88

9-
<div class="neuton" style="margin-bottom: 2em;">By <a href="/blog/{{page.nick}}">{{page.author}}</a> | {{page.date | date_to_string}}</div>
9+
<div class="neuton" style="margin-bottom: 2em;">By <a href="http://www.twitter.com/{{page.nick}}">{{page.author}}</a> | {{page.date | date_to_string}}</div>
1010

1111

1212
{{content}}

‎_layouts/clean.html

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{{ content }}

‎_layouts/default.html

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
<link href="//fonts.googleapis.com/css?family=Neuton:regular" rel="stylesheet" type="text/css" >
1414
<link rel="stylesheet" type="text/css" href="/styles/screen.css">
1515

16-
16+
<script type="text/javascript" src="/scripts/jquery-1.5.min.js"></script>
1717
<!-- Analytics -->
1818
<script type="text/javascript">
1919

‎_layouts/guide.html

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
layout: default
3+
---
4+
5+
<div id="guide">
6+
<div id="guide_content" style="float: left; width: 600px">
7+
{{ content }}
8+
</div>
9+
<div id="guide_sidebar" style="float: right; width: 120px">
10+
</div>
11+
<script>
12+
$('#guide_sidebar').load('sidebar.html');
13+
</script>
14+
</div>

‎_layouts/video.html

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
<a href="http://twitter.com/share" class="twitter-share-button" data-url="http://www.elasticsearch.org{{page.url}}" data-text="{{page.title}}" data-count="horizontal" data-via="elasticsearch" data-related="kimchy:Shay Banon: Founder of elasticsearch">Tweet</a><script type="text/javascript" src="http://platform.twitter.com/widgets.js"></script>
77
</div>
88

9-
<div class="neuton" style="margin-bottom: 2em;">By {{page.author}} | {{page.date | date_to_string}}</div>
9+
<div class="neuton" style="margin-bottom: 2em;">By <a href="http://www.twitter.com/{{page.nick}}">{{page.author}}</a> | {{page.date | date_to_string}}</div>
1010

1111

1212
{{content}}

‎blog/kimchy/index.html

-16
This file was deleted.

‎guide/index.html

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
layout: default
3+
title: Guide
4+
cat: guide
5+
title_in_header: false
6+
---
7+
8+
<div class="narrower">
9+
<h1>Part I: Introduction</h1>
10+
11+
<h1>Part II: Concepts</h1>
12+
13+
<h2><a href="">1. Analysis Paralysis</a></h2>
14+
15+
<h1>Part III: Reference</h1>
16+
17+
<h2><a href="/guide/reference/api">1. API</a></h2>
18+
19+
<h1>Part IV: Appendix</h1>
20+
21+
<h2><a href="#">1. Installing from Source</a></h2>
22+
</div>

‎guide/reference/api/bulk.textile

+58
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
layout: guide
3+
title: Bulk API
4+
cat: guide
5+
---
6+
7+
p. The bulk API allows to index and delete several documents in a single API. This can greatly increase the indexing speed.The REST API endpoint is /_bulk and it follows the following structure (for @json@):
8+
9+
<pre class="prettyprint">
10+
action_and_meta_data\n
11+
optional_source\n
12+
action_and_meta_data\n
13+
optional_source\n
14+
....
15+
action_and_meta_data\n
16+
optional_source\n
17+
</pre>
18+
19+
p. The json format relies on the fact that Json string values must have @\n@ escaped, and that the actual json actions and sources are not pretty printed. Here is an example:
20+
21+
<pre class="prettyprint">
22+
{ "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
23+
{ "type1" : { "field1" : "value1" } }
24+
{ "delete" : { "_index" : "test", "_type" : "type1", "_id" : "2" } }
25+
{ "create" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
26+
{ "field1" : "value1" }
27+
28+
</pre>
29+
30+
p. In the optional source part, the `type` is optional as is when indexing data.
31+
32+
p. A note on the format. The idea here is to make processing of this as fast as possible. As some of the actions will be needed to be redirected to other shards that exists on other nodes, only the action meta_data is parsed on the receiving node side. Also, zero copy buffers can be used on the source directly writing segments relevant to each action source to the network.
33+
34+
p. Client libraries using this protocol should try and strive to do something similar on the client side, and reduce as much as possible the creation of buffers.
35+
36+
p. The result is a full formatted json, with all the actions performed (in the same order), with possible error field indicating for each one in case of failure (on an item level).
37+
38+
p. Note, in the end, the full data needs to be represented on each server, so indexing 5GB of data should be broken down and not executed in a single batch.
39+
40+
p. If using the HTTP API, make sure that the client does not send HTTP chunks, as this will slow things down.
41+
42+
h1. Routing
43+
44+
p. Each bulk item can include the routing value using the @_routing@ field. It automatically follows the behavior of the index / delete operation based on the @_routing@ mapping.
45+
46+
h1. Parent
47+
48+
p. Each bulk item can include the parent value using the @_parent@ field. It automatically follows the behavior of the index / delete operation based on the @_parent@ / @_routing@ mapping.
49+
50+
h1. Write Consistency
51+
52+
p. Control if the operation will be allowed to execute based on the number of active shards within that partition (replication group). The values allowed are @one@, @quorum@, and @all@. The parameter to set it is @consistency@, and it defaults to the node level setting of @action.write_consistency@ which in turn defaults to @quorum@.
53+
54+
p. For example, in a N shards with 2 replicas index, there will have to be at least 2 active shards within the relevant partition (@quorum@) for the operation to succeed. In a N shards with 1 replica scenario, there will need to be a single shard active (in this case, @one@ and @quorum@ is the same).
55+
56+
h1. Refresh
57+
58+
p. The @refresh@ parameter can be set to @true@ in order to refresh the relevant shards after the bulk operation has occurred and make it searchable. Setting it to @true@ should be done after careful thought and verification that this does not cause a heavy load on the system (and slows down indexing).

‎guide/reference/api/count.textile

+79
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
layout: guide
3+
title: Index API
4+
cat: guide
5+
---
6+
7+
p. The count API allows to easily execute a query and get the number of matches for that query. It can be executed across one or more indices and across one or more types. The query can either be provided using a simple query string as a parameter, or using the "Query DSL":../query_dsl defined within the request body. Here is an example:
8+
9+
<pre class="prettyprint">
10+
$ curl -XGET 'http://localhost:9200/twitter/tweet/_count?q=user:kimchy'
11+
12+
$ curl -XGET 'http://localhost:9200/twitter/tweet/_count' -d '
13+
{
14+
"term" : { "user" : "kimchy" }
15+
}'
16+
</pre>
17+
18+
p. Both examples above end up doing the same thing, which is count the number of tweets from the twitter index for a certain user. The result of the commands is:
19+
20+
<pre class="prettyprint lang-js">
21+
{
22+
"count" : 1,
23+
"_shards" : {
24+
"total" : 5,
25+
"successful" : 5,
26+
"failed" : 0
27+
}
28+
}
29+
</pre>
30+
31+
h1. Multiple Indices and Types
32+
33+
p. The count API can be applied to multiple types within an index, and across multiple indices. For example, we can count all documents across all types that match a @user@ field with value @kimchy@:
34+
35+
<pre class="prettyprint">
36+
$ curl -XGET 'http://localhost:9200/_count?q=user:kimchy'
37+
</pre>
38+
39+
p. We can also count within specific types:
40+
41+
<pre class="prettyprint">
42+
$ curl -XGET 'http://localhost:9200/twitter/tweet,user/_count?q=user:kimchy'
43+
</pre>
44+
45+
p. We can also count all tweets with a certain tag across several indices (for example, when each user has his own index):
46+
47+
<pre class="prettyprint">
48+
$ curl -XGET 'http://localhost:9200/kimchy,elasticsearch/_count?q=tag:wow'
49+
</pre>
50+
51+
p. Or even count across all indices:
52+
53+
<pre class="prettyprint">
54+
$ curl -XGET 'http://localhost:9200/_count?q=tag:wow'
55+
</pre>
56+
57+
h1. Request Parameters
58+
59+
p. When executing count using the query parameter @q@, the query passed is a query string using Lucene query parser. There are additional parameters that can be passed:
60+
61+
|_. Name |_. Description |
62+
| @df@ | The default field to use when no field prefix is defined within the query. |
63+
| @analyzer@ | The analyzer name to be used when analyzing the query string. |
64+
| @default_operator@ | The default operator to be used, can be @AND@ or @OR@. Defaults to @OR@. |
65+
66+
h1. Request Body
67+
68+
p. The count can use the "Query DSL":../query_dsl within its body in order to express the query that should be executed. The body content can also be passed as a REST parameter named @source@.
69+
70+
p(note). Both HTTP GET and HTTP POST can be used to execute count with body. Since not all clients support GET with body, POST is allowed as well.
71+
72+
h1. Distributed
73+
74+
p. The count operation is broadcast across all shards. For each shard id group, a replica is chosen and executed against it. This means that replicas increase the scalability of count.
75+
76+
h1. Routing
77+
78+
p. The routing value (a comma separated list of the routing values) can be specified to control which shards the count request will be executed on.
79+

‎guide/reference/api/delete-by-query.textile

+90
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
---
2+
layout: guide
3+
title: Delete By Query API
4+
cat: guide
5+
---
6+
7+
p. The delete by query API allows to delete documents from one or more indices and one or more types based on a query. The query can either be provided using a simple query string as a parameter, or using the "Query DSL":../query_dsl defined within the request body. Here is an example:
8+
9+
<pre class="prettyprint">
10+
$ curl -XDELETE 'http://localhost:9200/twitter/tweet/_query?q=user:kimchy'
11+
12+
$ curl -XDELETE 'http://localhost:9200/twitter/tweet/_query' -d '{
13+
"term" : { "user" : "kimchy" }
14+
}
15+
'
16+
</pre>
17+
18+
p. Both above examples end up doing the same thing, which is delete all tweets from the twitter index for a certain user. The result of the commands is:
19+
20+
<pre class="prettyprint lang-js">
21+
{
22+
"ok" : true,
23+
"_indices" : {
24+
"twitter" : {
25+
"_shards" : {
26+
"total" : 5,
27+
"successful" : 5,
28+
"failed" : 0
29+
}
30+
}
31+
}
32+
}
33+
</pre>
34+
35+
h1. Multiple Indices and Types
36+
37+
p. The delete by query API can be applies to multiple types within an index, and across multiple indices. For example, we can delete all documents across all types within the twitter index:
38+
39+
<pre class="prettyprint">
40+
$ curl -XDELETE 'http://localhost:9200/twitter/_query?q=user:kimchy'
41+
</pre>
42+
43+
p. We can also delete within specific types:
44+
45+
<pre class="prettyprint">
46+
$ curl -XDELETE 'http://localhost:9200/twitter/tweet,user/_query?q=user:kimchy'
47+
</pre>
48+
49+
p. We can also delete all tweets with a certain tag across several indices (for example, when each user has his own index):
50+
51+
<pre class="prettyprint">
52+
$ curl -XDELETE 'http://localhost:9200/kimchy,elasticsearch/_query?q=tag:wow'
53+
</pre>
54+
55+
p. Or even delete across all indices:
56+
57+
<pre class="prettyprint">
58+
$ curl -XDELETE 'http://localhost:9200/_all/_query?q=tag:wow'
59+
</pre>
60+
61+
h1. Request Parameters
62+
63+
p. When executing a delete by query using the query parameter @q@, the query passed is a query string using Lucene query parser. There are additional parameters that can be passed:
64+
65+
|_. Name |_. Description |
66+
| @df@ | The default field to use when no field prefix is defined within the query. |
67+
| @analyzer@ | The analyzer name to be used when analyzing the query string. |
68+
| @default_operator@ | The default operator to be used, can be @AND@ or @OR@. Defaults to @OR@. |
69+
70+
h1. Request Body
71+
72+
p. The delete by query can use the "Query DSL":../query_dsl within its body in order to express the query that should be executed and delete all documents. The body content can also be passed as a REST parameter named @source@.
73+
74+
h1. Distributed
75+
76+
p. The delete by query API is broadcast across all primary shards, and from there, replicated across all shards replicas.
77+
78+
h1. Routing
79+
80+
p. The routing value (a comma separated list of the routing values) can be specified to control which shards the delete by query request will be executed on.
81+
82+
h1. Replication Type
83+
84+
p. The replication of the operation can be done in an asynchronous manner to the replicas (the operation will return once it has be executed on the primary shard). The @replication@ parameter can be set to @async@ (defaults to @sync@) in order to enable it.
85+
86+
h1. Write Consistency
87+
88+
p. Control if the operation will be allowed to execute based on the number of active shards within that partition (replication group). The values allowed are @one@, @quorum@, and @all@. The parameter to set it is @consistency@, and it defaults to the node level setting of @action.write_consistency@ which in turn defaults to @quorum@.
89+
90+
p. For example, in a N shards with 2 replicas index, there will have to be at least 2 active shards within the relevant partition (@quorum@) for the operation to succeed. In a N shards with 1 replica scenario, there will need to be a single shard active (in this case, @one@ and @quorum@ is the same).

0 commit comments

Comments
 (0)