Skip to content

Commit ae0b209

Browse files
author
Stanislaw Bogatkin
committed
Add main part of code.
Create first version with most options.
1 parent b55d6b5 commit ae0b209

20 files changed

+698
-0
lines changed

‎.fixtures.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
fixtures:
2+
repositories:
3+
'stdlib': 'https://github.com/puppetlabs/puppetlabs-stdlib.git'
4+
'concat': 'https://github.com/ripienaar/puppet-concat.git'
5+
symlinks:
6+
'twemproxy': "#{source_dir}"

‎Gemfile

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
source 'https://rubygems.org'
2+
3+
puppetversion = ENV.key?('PUPPET_VERSION') ? "#{ENV['PUPPET_VERSION']}" : ['>= 3.3']
4+
gem 'puppet', puppetversion
5+
gem 'puppetlabs_spec_helper', '>= 0.8.2'
6+
gem 'puppet-lint', '>= 1.0.0'
7+
gem 'facter', '>= 1.7.0'

‎README.md

+195
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
# twemproxy
2+
3+
#### Table of Contents
4+
5+
1. [Overview](#overview)
6+
2. [Module Description - What the module does and why it is useful](#module-description)
7+
3. [Setup - The basics of getting started with twemproxy](#setup)
8+
* [Beginning with twemproxy](#beginning-with-twemproxy)
9+
4. [Usage - Configuration options and additional functionality](#usage)
10+
5. [Reference - An under-the-hood peek at what the module is doing and how](#reference)
11+
5. [Limitations - OS compatibility, etc.](#limitations)
12+
6. [Development - Guide for contributing to the module](#development)
13+
14+
## Overview
15+
16+
It's a puppet module for installing and configuring Twemproxy - memcached proxy
17+
created in Twitter. Tested for:
18+
19+
* Ubuntu trusty
20+
21+
Should work for most linux distros.
22+
23+
## Module Description
24+
25+
This module install, configure and run twemproxy. You can do any of these tasks
26+
separately or combine them for installing twemproxy from scratch.
27+
28+
29+
## Setup
30+
31+
### Beginning with twemproxy
32+
33+
I believe that it should be enough to call
34+
35+
```puppet
36+
class { 'twemproxy':
37+
clients_array => $memcache_array,
38+
}
39+
```
40+
with array of `<memcache node address>:<port>:<weight>` values to start using
41+
twemproxy and force it to do something useful
42+
43+
## Usage
44+
45+
Most of the work can be done through main twemproxy class, so you can just call
46+
`::twemproxy` with options you need to use all functionality this module covers
47+
48+
###I need just setup twemproxy and create a pool with servers, what should I do?
49+
50+
```puppet
51+
class { 'twemproxy:
52+
clients_array => ['10.10.10.10:11211:1', '10.10.10.20:11211:1'],
53+
}
54+
```
55+
56+
###I need just manage already existing config
57+
58+
```puppet
59+
twemproxy::pool { 'mypool':
60+
listen_address => '127.0.0.1',
61+
listen_port => '22122',
62+
timeout => '400',
63+
redis => false,
64+
redis_auth => false,
65+
server_connections => 1,
66+
server_retry_timeout => 3000,
67+
clients_array => ['10.10.10.10:11211:1', '10.10.10.20:11211:1'],
68+
}
69+
```
70+
71+
## Reference
72+
73+
### Classes
74+
75+
* twemproxy: main class, include all others.
76+
* twemproxy::install: handles the package.
77+
* twemproxy::config: handles the daemon configuration file.
78+
* twemproxy::service: handles the service.
79+
* twemproxy::params: handles most of the parameters.
80+
81+
### Defined types
82+
83+
* twemproxy::listen: defines config part for twemproxy pool itself.
84+
* twemproxy::member: defines config part for memcache members in pool.
85+
* twemproxy::pool: combine previous two defines to one pool config.
86+
87+
### Parameters
88+
89+
Next parameter available in main twempxory class:
90+
91+
####`package_manage`
92+
Tells whether we should manage twemproxy package. Valid options: 'true' or
93+
'false'. Default option: 'true'.
94+
95+
####`package_name`
96+
Tells package name we should manage.
97+
98+
####`package_ensure`
99+
Set what we should do with package. Valid options: 'present' or 'absent'.
100+
Default value: 'present'.
101+
102+
####`service_manage`
103+
Tells if we should manage twemproxy service. Valid options: 'true' or 'false'.
104+
Default option: 'true'
105+
106+
####`service_name`
107+
Set service name for twemproxy.
108+
109+
####`service_ensure`
110+
Tells if we should install or remove service. Valid options: 'running' or
111+
'stopped'. Default option: 'running'
112+
113+
####`service_enable`
114+
Tells if we should enable service on system start. Valid options: 'true' or
115+
'false'. Default option: 'true'
116+
117+
####`listen_address`
118+
Address twemproxy will listen on. Default value: '127.0.0.1'
119+
120+
####`listen_port`
121+
Port twemproxy will listen on. Default value: 22122
122+
123+
####`timeout`
124+
The timeout value in msec that twemproxy will wait for to establish a connection
125+
o the server or receive a response from a server. Set it to false if you want
126+
to wait indefinitely. Default value: 400
127+
128+
####`backlog`
129+
The TCP backlog argument. Default value: 1024
130+
131+
####`preconnect`
132+
A boolean value that controls if twemproxy should preconnect to all the servers
133+
in this pool on process start. Default value: 'false'
134+
135+
####`redis`
136+
A boolean value that controls if a server pool speaks redis or memcached
137+
protocol. Default value: 'false'
138+
139+
####`redis_auth`
140+
Authenticate to the Redis server on connect. Default value: 'false'
141+
142+
####`redis_db`
143+
The DB number to use on the pool servers. Defaults to false
144+
145+
####`server_connections`
146+
The maximum number of connections that can be opened to each server. Default
147+
value: 1
148+
149+
####`auto_eject_hosts`
150+
A boolean value that controls if server should be ejected temporarily when it
151+
fails consecutively `server_failure_limit` times. Default value: 'false'
152+
153+
####`server_retry_timeout`
154+
The timeout value in msec to wait for before retrying on a temporarily ejected
155+
server, when `auto_eject_host` is set to true. Default value: 30000
156+
157+
####`server_failure_limit`
158+
The number of consecutive failures on a server that would lead to it being
159+
temporarily ejected when `auto_eject_host` is set to true. Default value: 2
160+
161+
####`client_address`
162+
Address of client for twemproxy.
163+
164+
####`client_port`
165+
Port which client will bind to. Default value: 11211
166+
167+
####`client_weight`
168+
Weight of client. Default value: 1
169+
170+
####`clients_array`
171+
Array with client conections. You can consider it like array with different
172+
clients pointed as '`client_address`:`client_port`:`client_weight`'. Example
173+
of use:
174+
175+
```puppet
176+
[
177+
'10.10.10.10:11211:1',
178+
'10.10.20.20:11211:1',
179+
'10.10.30.30:11211:2',
180+
]
181+
```
182+
183+
## Limitations
184+
185+
This module was tested only on Ububtu trusty, but should probably work on
186+
other platforms too.
187+
188+
## Development
189+
190+
This module is completely free, so I would appreciate any pull-requests to it
191+
or its redistribution.
192+
193+
## Contributors
194+
195+
- Stanislaw Bogatkin

‎Rakefile

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
require 'puppetlabs_spec_helper/rake_tasks'
2+
require 'puppet-lint/tasks/puppet-lint'
3+
PuppetLint.configuration.send('disable_80chars')
4+
PuppetLint.configuration.ignore_paths = ["spec/**/*.pp", "pkg/**/*.pp"]
5+
6+
desc "Validate manifests, templates, and ruby files"
7+
task :validate do
8+
Dir['manifests/**/*.pp'].each do |manifest|
9+
sh "puppet parser validate --noop #{manifest}"
10+
end
11+
Dir['spec/**/*.rb','lib/**/*.rb'].each do |ruby_file|
12+
sh "ruby -c #{ruby_file}" unless ruby_file =~ /spec\/fixtures/
13+
end
14+
Dir['templates/**/*.erb'].each do |template|
15+
sh "erb -P -x -T '-' #{template} | ruby -c"
16+
end
17+
end

‎manifests/config.pp

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
class twemproxy::config inherits twemproxy {
2+
3+
if $twemproxy::package_manage {
4+
file { '/etc/default/twemproxy':
5+
owner => 'root',
6+
group => 'root',
7+
mode => '0644',
8+
content => template('twemproxy/config.erb'),
9+
}
10+
}
11+
}

‎manifests/init.pp

+67
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Class: twemproxy
2+
# ===========================
3+
#
4+
# It is a main class to install and configure twemproxy
5+
#
6+
#
7+
# Examples how to use
8+
# -------------------
9+
#
10+
# class { 'twemproxy':
11+
# clients_array => [ '10.10.10.10:11211:1', '10.10.10.20:11211:1' ],
12+
# }
13+
#
14+
# Authors
15+
# -------
16+
#
17+
# Stanislaw Bogatkin <[email protected]>
18+
#
19+
#
20+
class twemproxy (
21+
$package_manage = $twemproxy::params::package_manage,
22+
$package_name = $twemproxy::params::package_name,
23+
$package_ensure = $twemproxy::params::package_ensure,
24+
$service_manage = $twemproxy::params::service_manage,
25+
$service_name = $twemproxy::params::service_name,
26+
$service_ensure = $twemproxy::params::service_ensure,
27+
$service_enable = $twemproxy::params::service_enable,
28+
$listen_address = $twemproxy::params::listen_address,
29+
$listen_port = $twemproxy::params::listen_port,
30+
$timeout = $twemproxy::params::timeout,
31+
$backlog = $twemproxy::params::backlog,
32+
$preconnect = $twemproxy::params::preconnect,
33+
$redis = $twemproxy::params::redis,
34+
$redis_auth = $twemproxy::params::redis_auth,
35+
$redis_db = $twemproxy::params::redis_db,
36+
$server_connections = $twemproxy::params::server_connections,
37+
$auto_eject_hosts = $twemproxy::params::auto_eject_hosts,
38+
$server_retry_timeout = $twemproxy::params::server_retry_timeout,
39+
$server_failure_limit = $twemproxy::params::server_failure_limit,
40+
$client_address = undef,
41+
$client_port = $twemproxy::params::client_port,
42+
$client_weight = $twemproxy::params::client_weight,
43+
$clients_array = undef,
44+
) inherits twemproxy::params {
45+
46+
anchor { 'twemproxy::start': } ->
47+
class { 'twemproxy::install': } ->
48+
class { 'twemproxy::config': } ->
49+
50+
twemproxy::pool { 'default':
51+
listen_address => $listen_address,
52+
listen_port => $listen_port,
53+
timeout => $timeout,
54+
redis => $redis,
55+
redis_auth => $redis_auth,
56+
server_connections => $server_connections,
57+
server_retry_timeout => $server_retry_timeout,
58+
client_address => $client_address,
59+
client_port => $client_port,
60+
client_weight => $client_weight,
61+
clients_array => $clients_array,
62+
} ->
63+
64+
class { 'twemproxy::service': } ->
65+
anchor { 'twemproxy::end': }
66+
67+
}

‎manifests/install.pp

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
class twemproxy::install inherits twemproxy {
2+
3+
if $twemproxy::package_manage {
4+
package { $twemproxy::package_name:
5+
ensure => $twemproxy::package_ensure,
6+
}
7+
}
8+
}

‎manifests/listen.pp

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
define twemproxy::listen (
2+
$order,
3+
$listen_address,
4+
$listen_port,
5+
$hash,
6+
$distribution,
7+
$auto_eject_hosts = false,
8+
$backlog = undef,
9+
$connections = undef,
10+
$hash_tag = undef,
11+
$preconnect = false,
12+
$redis = false,
13+
$redis_auth = false,
14+
$redis_db = false,
15+
$server_connections = undef,
16+
$server_retry_timeout = undef,
17+
$server_failure_limit = undef,
18+
$timeout = undef,
19+
) {
20+
21+
concat { '/etc/nutcracker.yml':
22+
owner => 'root',
23+
group => 'root',
24+
mode => '0644',
25+
}
26+
27+
concat::fragment { "${name}_listen_block":
28+
order => "00_${order}",
29+
target => '/etc/nutcracker.yml',
30+
content => template('twemproxy/twemproxy_listen_block.erb'),
31+
}
32+
}

‎manifests/member.pp

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
define twemproxy::member (
2+
$order,
3+
$client_address = undef,
4+
$client_port = undef,
5+
$client_weight = undef,
6+
$clients_array = undef,
7+
) {
8+
9+
if $client_address and $clients_array {
10+
fail('clients_array is mutual exclusive with client_address parameter')
11+
}
12+
13+
concat::fragment { "${name}_member_block":
14+
order => "01_${order}",
15+
target => '/etc/nutcracker.yml',
16+
content => template('twemproxy/twemproxy_member_block.erb'),
17+
}
18+
}

0 commit comments

Comments
 (0)