Repository navigation
Expand file tree
/
Copy pathparams.json
More file actions
1 lines (1 loc) · 17 KB
/
Copy pathparams.json
File metadata and controls
1 lines (1 loc) · 17 KB
1
{"name":"Cqerl","tagline":"Native Erlang CQL driver","body":"# CQErl\r\n\r\nNative Erlang client for CQL3 over Cassandra's binary protocol v2 (a.k.a. what you want as a client for Cassandra).\r\n\r\n[**Usage**](#usage) · [Connecting](#connecting) · [Performing queries](#performing-queries) · [Query options](#providing-options-along-queries) · [Batched queries](#batched-queries) · [Reusable queries](#reusable-queries) · [Data types](#data-types)\r\n\r\n[**Installation**](#installation) · [**Tests**](#tests) · [**License**](#license)\r\n\r\n---\r\n\r\n*This project is still relatively new and is under active development, so you are welcome to contribute to it with improvements, suggestions and any issues you encounter.*\r\n\r\n### At a glance\r\n\r\nCQErl offers a simple Erlang interface to Cassandra using the latest CQL version (v3). The main features include:\r\n\r\n* Automatic (and configurable) connection pools using [pooler][1]\r\n* Batched queries\r\n* Variable bindings in CQL queries (named or not)\r\n* Automatic query reuse when including variable bindings\r\n* Collection types support\r\n* Tunable consistency level\r\n* Synchronous or asynchronous queries\r\n* Automatic compression (using lz4 or snappy if available)\r\n* SSL support\r\n* Pluggable authentication (as long as it's [SASL][2]-based)\r\n\r\nCQErl was designed to be as simple as possible on your side. You just provide the configuration you want as environment variables, and ask for a new client everytime you need to perform a transient piece of work (e.g. handle a web request). You do not need to (and should not) keep a client in state for a long time. Under the hood, CQErl maintains a pool of persistent connections with Cassandra and this pattern is the best way to ensure proper load balancing of requests across the pool.\r\n\r\n### Usage\r\n\r\n##### Connecting\r\n\r\nIf you installed cassandra and didn't change any configuration related to authentication or SSL, you should be able to connect like this\r\n\r\n```erlang\r\n{ok, Client} = cqerl:new_client({}).\r\n```\r\n \r\nAnd close the connection like this\r\n\r\n```erlang\r\ncqerl:close_client(Client).\r\n```\r\n\r\n1. The first argument to `cqerl:new_client/2` or `cqerl_new_client/1` is the node to which you wish to connect as `{Ip, Port}`. If empty, it defaults to `{\"127.0.0.1\", 9042}`, and `Ip` can be given as a string, or as a tuple of components, either IPv4 or IPv6.\r\n\r\n2. The second possible argument (when using `cqerl:new_client/2`) is a list of options, that include `auth` (mentionned below), `ssl` (which is `false` by default, but can be set to a list of SSL options) and `keyspace` (string or binary). Other options include `pool_max_size`, `pool_min_size` and `pool_cull_interval`, which are used to configure [pooler][1] (see its documentation to understand those options).\r\n\r\nIf you've set simple username/password authentication scheme on Cassandra, you can provide those to CQErl\r\n\r\n```erlang\r\n{ok, Client} = cqerl:new_client({}, [{auth, {cqerl_auth_plain_handler, [{\"test\", \"aaa\"}]}}]).\r\n```\r\n \r\nSince Cassandra implements pluggable authentication mechanisms, CQErl also allows you to provide custom authentication modules (here `cqerl_auth_plain_handler`). The options you pass along with it are given to the module's `auth_init/3` as its first argument.\r\n\r\n###### Using environment variables\r\n\r\nAll the options given above can be provided as environment variables, in which case they are used as default (and overridable) values to any `cqerl:new_client` calls. You can also provide a `cassandra_nodes` variable containing a list of the tuples used as the first argument to `cqerl:new_client`. So for example, in your `app.config` or `sys.config` file, you could have the following content:\r\n\r\n```erlang\r\n[\r\n {cqerl, [\r\n {cassandra_nodes, [ { \"127.0.0.1\", 9042 } ]},\r\n {ssl, [ {cacertfile, \"cassandra.pem\"} ]},\r\n {auth, {cqerl_auth_plain_handler, [ {\"test\", \"aaa\"} ]}}\r\n ]},\r\n]\r\n```\r\n\r\nDoing so will fire up connection pools as soon as the CQErl application is started. So when later on you call `cqerl:new_client`, chances are you will hit a preallocated connection (unless they're so busy that CQErl needs to fire up new ones). In fact, if you provide the `cassandra_nodes` environment variable, you can call `cqerl:get_client/0`, which chooses an available client at random.\r\n\r\n##### Performing queries\r\n\r\nPerforming a query can be as simple as this:\r\n\r\n```erlang\r\n{ok, Result} = cqerl:run_query(Client, \"SELECT * FROM users;\").\r\n\r\n% Equivalent to\r\n{ok, Result} = cqerl:run_query(Client, <<\"SELECT * FROM users;\">>).\r\n\r\n% Also equivalent to\r\n{ok, Result} = cqerl:run_query(Client, #cql_query{statement = <<\"SELECT * FROM users;\">>}).\r\n```\r\n\r\nIt can also be performed asynchronously using\r\n\r\n```erlang\r\nTag = cqerl:send_query(Client, \"SELECT * FROM users;\"),\r\nreceive\r\n {result, Tag, Result} ->\r\n ok\r\nend.\r\n```\r\n\r\nIn situations where you do not need to wait for the response at all, it's perfectly fine to produce this sort of pattern:\r\n\r\n```erlang\r\n{ok, Client} = cqerl:new_client(),\r\ncqerl:send_query(Client, #cql_query{statement=\"UPDATE secrets SET value = null WHERE id = ?;\",\r\n values=[{id, <<\"42\">>}]}),\r\ncqerl:close_client(Client).\r\n```\r\n\r\nThat is, you can grab a client only the send a query, then you can get rid of it. CQErl will still perform it,\r\nthe difference being that no response will be sent back to you.\r\n\r\nHere's a rundown of the possible return values\r\n\r\n* `SELECT` queries will yield result of type `#cql_result{}` (more details below). \r\n* Queries that change the database schema will yield result of type `#cql_schema_changed{type, keyspace, table}`\r\n* Other queries will yield `void` if everything worked correctly.\r\n* In any case, errors returned by cassandra in response to a query will be the return value (`{error, Reason}` in the synchronous case, and `{error, Tag, Reason}` in the asynchronous case).\r\n\r\n###### `#cql_result{}`\r\n\r\nThe return value of `SELECT` queries will be a `#cql_result{}` record, which can be used to obtain rows as proplists and fetch more result if available\r\n\r\n```erlang\r\n{ok, _SchemaChange} = cqerl:run_query(Client, \"CREATE TABLE users(id uuid, name varchar, password varchar);\"),\r\n{ok, void} = cqerl:run_query(Client, #cql_query{\r\n statement = \"INSERT INTO users(id, name, password) VALUES(?, ?, ?);\",\r\n values = [\r\n {id, new},\r\n {name, \"matt\"},\r\n {password, \"qwerty\"}\r\n ]\r\n}),\r\n{ok, Result} = cqerl:run_query(Client, \"SELECT * FROM users;\").\r\n\r\nRow = cqerl:head(Result),\r\nTail = cqerl:tail(Result),\r\n{Row, Tail} = cqerl:next(Result),\r\n1 = cqerl:size(Result),\r\n0 = cqerl:size(Tail),\r\nempty_dataset = cqerl:next(Tail),\r\n[Row] = cqerl:all_rows(Result),\r\n\r\n<<\"matt\">> = proplists:get_value(name, Row),\r\n<<\"qwerty\">> = proplists:get_value(password, Row).\r\n```\r\n\r\n`#cql_result{}` can also be used to fetch more result, synchronously or asynchronously\r\n\r\n```erlang\r\n\r\ncase cqerl:has_more_pages(Result) of\r\n true -> {ok, Result2} = cqerl:fetch_more(Result);\r\n false -> ok\r\nend,\r\n\r\nTag2 = cqerl:fetch_more_async(Result),\r\nreceive\r\n {result, Tag2, Result2} -> ok\r\nend.\r\n```\r\n\r\n###### `#cql_schema_changed{}`\r\n\r\n`#cql_schema_changed{}` is returned from queries that change the database schema somehow (e.g. `ALTER`, `DROP`, `CREATE`, and so on). It includes:\r\n\r\n1. The `type` of change, either `created`, `updated` or `dropped`\r\n2. The name of the `keyspace` where the change happened, as a binary\r\n3. If applicable, the name of `table` on which the change was applied, as a binary\r\n\r\n##### Providing options along queries\r\n\r\nWhen performing queries, you can provide more information than just the query statement using the `#cql_query{}` record, which includes the following fields:\r\n\r\n1. The query `statement`, as a string or binary\r\n2. `values` for binding variables from the query statement (see next section).\r\n \r\n3. You can tell CQErl to consider a query `reusable` or not (see below for what that means). By default, it will detect binding variables and consider it reusable if it contains (named or not) any. Queries containing *named* binding variables will be considered reusable no matter what you set `reusable` to. If you explicitely set `reusable` to `false` on a query having positional variable bindings (`?`), you would provide values with in `{Type, Value}` pairs instead of `{Key, Value}`. \r\n4. You can specify how many rows you want in every result page using the `page_size` (integer) field. The devs at Cassandra recommend a value of 100 (which is the default).\r\n5. You can also specify what `consistency` you want the query to be executed under. Possible values include (all defined in [`include/cqerl.hrl`](include/cqerl.hrl)):\r\n\r\n * `?CQERL_CONSISTENCY_ANY`\r\n * `?CQERL_CONSISTENCY_ONE` \r\n * `?CQERL_CONSISTENCY_TWO` \r\n * `?CQERL_CONSISTENCY_THREE`\r\n * `?CQERL_CONSISTENCY_QUORUM` \r\n * `?CQERL_CONSISTENCY_ALL` \r\n * `?CQERL_CONSISTENCY_LOCAL_QUORUM`\r\n * `?CQERL_CONSISTENCY_EACH_QUORUM`\r\n * `?CQERL_CONSISTENCY_LOCAL_ONE`\r\n \r\n6. In case you want to perform a [lightweight transaction][4] using `INSERT` or `UPDATE`, you can also specify the `serial_consistency` that will be use when performing it. Possible values are:\r\n\r\n * `?CQERL_CONSISTENCY_SERIAL`\r\n * `?CQERL_CONSISTENCY_LOCAL_SERIAL`\r\n \r\n##### Variable bindings\r\n\r\nIn the `#cql_query{}` record, you can provide `values` as a `proplists`, where the keys match the column names or binding variable names in the statement, in **lowercase**. \r\n\r\nSpecial cases include: \r\n\r\n- providing `TTL` and `TIMESTAMP` option in statements, in which case the proplist key would be `[ttl]` and `[timestamp]` respectively. Note that, while values for a column of type `timestamp` are provided in **milliseconds**, a value for the `TIMESTAMP` option is expected in **microseconds**.\r\n- `UPDATE keyspace SET set = set + ? WHERE id = 1;`. The name for this variable binding is `set`, the name of the column, and it's expected to be an erlang **list** of values.\r\n- `UPDATE keyspace SET list = list + ? WHERE id = 1;`. The name for this variable binding is `list`, the name of the column, and it's expected to be an erlang **list** of values.\r\n- `UPDATE keyspace SET map[?] = 1 WHERE id = 1;`. The name for this variable binding is `key(map)`, where `map` is the name of the column.\r\n- `UPDATE keyspace SET map['key'] = ? WHERE id = 1;`. The name for this variable binding is `value(map)`, where `map` is the name of the column.\r\n- `UPDATE keyspace SET list[?] = 1 WHERE id = 1;`. The name for this variable binding is `idx(list)`, where `list` is the name of the column.\r\n\r\n Also, when providing the value for a `uuid`-type column, you can give the value `new`, `strong` or `weak`, in which case CQErl will generate a random UUID (v4), with either a *strong* or *weak* number random generator.\r\n \r\n Finally, when providing the value for a `timeuuid` or `timestamp` column, you can give the value `now`, in which case CQErl will generate a normal timestamp, or a UUID (v1) matching the current date and time.\r\n\r\n##### Batched queries\r\n\r\nTo perform batched queries (which can include any non-`SELECT` [DML][5] statements), simply put one or more `#cql_query{}` records in a `#cql_query_batch{}` record, and run it in place of a normal `#cql_query{}`. `#cql_query_batch{}` include the following fields:\r\n\r\n1. The `consistency` level to apply when executing the batch of queries.\r\n2. The `mode` of the batch, which can be `?CQERL_BATCH_LOGGED`, `?CQERL_BATCH_UNLOGGED` or `?CQERL_BATCH_COUNTER` (declared in [`include/cqerl.hrl`](include/cqerl.hrl)). Running a batch in *unlogged* mode removes the performance penalty of enforcing atomicity. The *counter* mode should be used to perform batched mutation of counter values.\r\n3. Finally, you must specify the list of `queries`.\r\n\r\n```erlang\r\nInsertQ = #cql_query{statement = \"INSERT INTO users(id, name, password) VALUES(?, ?, ?);\"},\r\n{ok, void} = cqerl:run_query(Client, #cql_query_batch{\r\n mode=?CQERL_BATCH_UNLOGGED,\r\n queries=[\r\n InsertQ#cql_query{values = [{id, new},{name, \"sean\"},{password, \"12312\"}]},\r\n InsertQ#cql_query{values = [{id, new},{name, \"jenna\"},{password, \"11111\"}]},\r\n InsertQ#cql_query{values = [{id, new},{name, \"kate\"},{password, \"foobar\"}]}\r\n ]\r\n}).\r\n```\r\n\r\n##### Reusable queries\r\n\r\nIf any of the following is true:\r\n\r\n* you set `#cql_query{}`'s `reusable` field to `true`\r\n* the query contains positional variable bindings (`?`) and you did not explicitely `reusable` to false\r\n* the query contains named variable bindings (`:name`) (ignores the value of `reusable`)\r\n\r\nthe query is considered *reusable*. This means that the first time this query will be performed, CQErl will ask the connected Cassandra node to prepare the query, after which, internally, a query ID will be used instead of the query statement when executing it. That particular cassandra node will hold on to the prepared query on its side and subsequent queries *that use exactly the same statement* [will be performed faster and with less network traffic][7].\r\n\r\nCQErl can tell which query has been previously prepared on which node by keeping a local cache, so all of this happens correctly and transparently.\r\n\r\n##### Data types\r\n\r\nHere is a correspondance of cassandra column types with their equivalent Erlang types (bold denotes what will used in result sets, the rest is what is accepted).\r\n\r\nCassandra Column Type | Erlang types\r\n----------------------|-----------------\r\nascii | **binary**, string (only US-ASCII)\r\nbigint | **integer** (signed 64-bit)\r\nblob | **binary**\r\nboolean | `true`, `false`\r\ncounter | **integer** (signed 64-bit)\r\ndecimal | `{Unscaled :: integer(), Scale :: integer()}`\r\ndouble | **float** (signed 64-bit)\r\nfloat | **float** (signed 32-bit)\r\nint | **integer** (signed 32-bit)\r\ntimestamp | **integer** (milliseconds, signed 64-bit), `now`, [binary or string][6]\r\nuuid | **binary**, `new`\r\nvarchar | **binary**, string\r\nvarint | **integer** (arbitrary precision)\r\ntimeuuid | **binary**, `now`\r\ninet | `{X1, X2, X3, X4}` (IPv4), `{Y1, Y2, Y3, Y4, Y5, Y6, Y7, Y8}` (IPv6), string or binary\r\n\r\n### Installation\r\n\r\nJust include this repository in your project's `rebar.config` file and run `./rebar get-deps`. See [rebar][3] for more details on how to use rebar for Erlang project management.\r\n\r\n### Tests\r\n\r\nCQErl includes a test suite that you can run yourself, especially if you plan to contribute to this project. \r\n\r\n1. Clone this repo on your machine\r\n2. Edit `test/test.config` and put your own cassandra's configurations\r\n3. At the project's top directory, run `make test`\r\n\r\n### License\r\n\r\nThe MIT License (MIT)\r\n\r\nCopyright (c) 2013 Mathieu D'Amours\r\n\r\nPermission is hereby granted, free of charge, to any person obtaining a copy of\r\nthis software and associated documentation files (the \"Software\"), to deal in\r\nthe Software without restriction, including without limitation the rights to\r\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of\r\nthe Software, and to permit persons to whom the Software is furnished to do so,\r\nsubject to the following conditions:\r\n\r\nThe above copyright notice and this permission notice shall be included in all\r\ncopies or substantial portions of the Software.\r\n\r\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\r\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS\r\nFOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR\r\nCOPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER\r\nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r\nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\r\n\r\n[1]: https://github.com/seth/pooler\r\n[2]: http://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer\r\n[3]: https://github.com/rebar/rebar\r\n[4]: http://www.datastax.com/documentation/cassandra/2.0/webhelp/index.html#cassandra/dml/dml_about_transactions_c.html\r\n[5]: http://en.wikipedia.org/wiki/Data_manipulation_language\r\n[6]: http://www.datastax.com/documentation/cql/3.0/webhelp/index.html#cql/cql_reference/cql_data_types_c.html#reference_ds_dsf_555_yj\r\n[7]: http://www.datastax.com/dev/blog/client-side-improvements-in-cassandra-2-0\r\n","google":"","note":"Don't delete this file! It's used internally to help with page regeneration."}