Vladimir Smirnov 5cebdc9382
Merge pull request #62 from gunnihinn/master
2 weeks ago
cache remove memcache support 1 year ago
contrib Fix ExecStart to use correct env var 1 year ago
internal/ipb3 internal/ipb3: internal protobuf definitions 1 year ago
intervalset add pickle support for graphtie-web 1.0 1 year ago
limiter Make split by globs working 2 months ago
mlog mlog: update for upstream rotatelogs API change 1 year ago
mstats mstats: add TotalAlloc 2 years ago
pathcache A lot of bugfixes in new code 2 months ago
util Remove carbonapi cross-dependency 8 months ago
vendor Add auto protocol 1 month ago
zipper Setup an empty response to merge into 2 weeks ago
.gitignore Commit vendor folder 2 months ago
.travis.yml Temporary disable package builds 2 months ago
CHANGES.md Add support for msgpack protocol 1 month ago
Gopkg.lock Add auto protocol 1 month ago
Gopkg.toml Add auto protocol 1 month ago
LICENSE Initial tests for BroadcastGroup 3 weeks ago
Makefile Add "install" target to Makefile 1 year ago
README.md Initial tests for BroadcastGroup 3 weeks ago
example.conf Add support for msgpack protocol 1 month ago
grpc.go Major rework of carbonzipper balancer logic 2 months ago
main.go Migrate carbonzipper to viper as config parsing library 1 month ago

README.md

carbonzipper: carbonserver proxy for graphite-web

Build Status

We are using to host our packages!

Please note that since carbonapi 0.8 it's no longer needed to run separate carbonzipper. This repo still contains a buildable daemon, but it's mostly for compatibility reasons and for those who don't need carbonapi functionality.

CarbonZipper is the central part of a replacement graphite storage stack. It proxies requests from graphite-web to a cluster of carbon storage backends. Previous versions (available in the git history) were able to talk to python carbon stores, but the current version requires the use of go-carbon or graphite-clickhouse.

Installation

Stable versions: Stable repo

Autobuilds (master, might be unstable): Autobuild repo

General information

Configuration is done via a YAML file loaded at startup. The only required field is the list of carbonserver backends to connect to.

Other pieces of the stack are:

For an overview of the stack and how the pieces fit together, watch Graphite@Scale or How to store millions metrics per second from FOSDEM '17 or Graphite@Scale or How to Store Millions of metrics per Second from SRECon17 Asia.

Requirements

carbonzipper requires Go 1.8+ to build. It's recommended to always use latest stable.

CarbonZipper uses protobuf-based protocol to talk with underlying storages. For current version the compatibility list is:

  1. carbonzipper >= 0.50
  2. go-carbon >= 0.9.0 (Note: you need to enable carbonserver in go-carbon).
  3. carbonserver@master (Note: you should probably switch to go-carbon in that case).
  4. graphite-clickhouse any. That's alternative storage that doesn't use Whisper. Limitations: /info handler won't work properly.
  5. carbonapi >= 0.5. Note: we are not sure if there is any point in running carbonzipper over carbonapi at this moment.

Changes and versioning

Version policy - all the versions we run in production is taged.

In case change will require simultanious upgrade of different components, it will be stated in Upgrading notes below.

Also we will try to maintain backward compatibility from down to up.

For example with protobuf2 -> protobuf3 migration - carbonzipper can still send results to older versions of carbonapi, but can't get results from older versions of carbonserver/go-carbon.

See CHANGES.md

Upgrading to 0.60 from 0.50 or earlier

Starting from 0.60, carbonzipper will be able to talk only with storages compatible with protobuf3.

At this moment (0.60) it's only go-carbon, starting from commit ee2bc24 (post 0.9.1)

Carbonzipper can still return results in protobuf and compatibility won't be removed at least until Summer 2017.

If you want to upgrade, the best option is to do follwing steps:

  1. Migrate to go-carbon post 0.9.1 release. (note: carbonserver isn't compatible with this version of zipper)
  2. Migrate to carbonsearch 0.16.0 (if you are using any)
  3. Upgrade carbonzipper to 0.60 or newer.
  4. Upgrade carbonapi to 0.6.0 (commit 119e346 or newer) (optional, but advised)

Acknowledgement

This program was originally developed for Booking.com. With approval from Booking.com, the code was generalised and published as Open Source on github, for which the author would like to express his gratitude.

License

This code is licensed under the MIT license.

Contact

If you have questions or problems there are two ways to contact us:

  1. Open issue on a github page
  2. #zipperstack on gophers slack