On 3/27/12 3:22 PM, Jason Penton wrote:

wow Daniel, you've been busy ;)

nice!!!!

thanks, still a bit to tune it (add the feature of enable/disable connections), but should be workable as it is. Couldn't test all scenarios, so feedback would be welcome!

Cheers, Daniel

On Tue, Mar 27, 2012 at 2:44 PM, Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> wrote:

Module: sip-router
Branch: master
Commit: 201fc2d600e48fbb717531c79013c1b971f82d76
URL:
http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=201fc2d600e48fbb717531c79013c1b971f82d76

Author: Daniel-Constantin Mierla <miconda@gmail.com
<mailto:miconda@gmail.com>>
Committer: Daniel-Constantin Mierla <miconda@gmail.com
<mailto:miconda@gmail.com>>
Date:   Tue Mar 27 14:38:57 2012 +0200

db_cluster: new module for generic database clustering

- the module allow to group several DB connections and perform
commands
 over them, based on policies such as round robin, serial try or
 parallel execution
- any kind of db connection can make part from a cluster
- example use case: write in parallel to two databases and do round
 robin read from them
- the module is a middle layer between modules and databases, reusing
 existing db connectors. A module that wants to use such cluster, just
 have to set the db_url to "cluster://clusterid"
- the module allow definition of multiple clusters
- see README for more details
- todo: enable/disable connections at runtime and via rpc

---

 modules_k/db_cluster/Makefile                 |   15 +
 modules_k/db_cluster/README                   |  162 +++++++++
 modules_k/db_cluster/db_cluster_mod.c         |  112 ++++++
 modules_k/db_cluster/dbcl_api.c               |  437
++++++++++++++++++++++
 modules_k/db_cluster/dbcl_api.h               |  142 ++++++++
 modules_k/db_cluster/dbcl_data.c              |  479
+++++++++++++++++++++++++
 modules_k/db_cluster/dbcl_data.h              |   89 +++++
 modules_k/db_cluster/doc/Makefile             |    4 +
 modules_k/db_cluster/doc/db_cluster.xml       |   37 ++
 modules_k/db_cluster/doc/db_cluster_admin.xml |  167 +++++++++
 10 files changed, 1644 insertions(+), 0 deletions(-)

Diff:
http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commitdiff;h=201fc2d600e48fbb717531c79013c1b971f82d76

_______________________________________________
sr-dev mailing list
sr-dev@lists.sip-router.org <mailto:sr-dev@lists.sip-router.org>
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev

---

sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev

Hello,

On 3/29/12 7:33 PM, Juha Heinanen wrote:

daniel,

i gave db_cluster module a try. i configured serial connections and it worked fine: when i stopped mysql server behind the higher priority connection, queries started to work over the lower priority connection.

there is one thing that bothers me. when higher priority connection is not available, i get these kind of error to syslog even when queries succeed over the lower priority connection:

Mar 29 20:24:52 sip /usr/sbin/sip-proxy[18966]: ERROR: db_mysql [km_dbase.c:122]: driver error on query: Can't connect to MySQL server on '192.98.102.13' (115) Mar 29 20:24:52 sip /usr/sbin/sip-proxy[18966]: ERROR:<core> [db_query.c:156]: error while submitting query

it would be nice if those error messages could somehow be suppressed when one of the connections is still operational and the queries actually succeed. someone may namely run such incomplete cluster many hours and during that time syslog will be filled with those (confusing) error messages.

maybe we should introduce a log_level parameter for mysql and the other db modules that can be controlled via config/rpc.

On the other hand, I still have some items in to-do list for the module. The idea is to disable a connection if a query fails and try to reconnect either upon a rpc command or after some time or a number skipped queries. At that time the error should appear only once, at the moment the problem is detected, eventually repeating from time to time when the module tries to auto-reconnect. Other suggestions are welcome, of course!

Cheers, Daniel

Daniel-Constantin Mierla writes:

Perhaps would be faster either to extend domain(s) to detect DB API version and use the one available or enhance domain(k) to offer same things as domain(s). AFAIK for the second, its about having a domain id in domain table and eventually loading domain attributes.

daniel,

i added support for did and domain attributes to modules_k/domain. i'll update new in 3.3 wiki page during the next couple of days.

-- juha

Hello,

On 3/30/12 11:47 AM, Marius Zbihlei wrote:

On 03/27/2012 03:44 PM, Daniel-Constantin Mierla wrote:

...

Module: sip-router Branch: master Commit: 201fc2d600e48fbb717531c79013c1b971f82d76 URL: http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=201fc2d6...

Author: Daniel-Constantin Mierlamiconda@gmail.com Committer: Daniel-Constantin Mierlamiconda@gmail.com Date: Tue Mar 27 14:38:57 2012 +0200

Hello Daniel,

I have a few questions regarding the db_cluster module and especially the way it deals with errors:

For serial operation , lets consider two handlers DB1 and DB2 with the same priority. For the first write operation DB1 lets presume that itfails so the insert is done on DB2 (network congestion, mysql deadlock etc). But for a serial select the first DB1 is chosen (I looked thru the code and I see no ways of caching the initial error) and this means that the info returned from DB1 (it might be insert_update or update so info might be also present in DB1). How does the module handle this?

Same scenarios and question I think it applies with round-robin mode as well.

The way we do this in p_usrloc is by keeping a error counter per each handler that is associated with a state (on -off) and a timestamp (when it failed). This info can be used to disable usage of the DB handler( and later put the handler in Write-Only mode until the data is synchronized )

error detection and connection (auto-) enable/disable are not implemented yet. They were in mind, but couldn't decide quickly what solution to do.

Each connection links to a structure in shared memory - dbcl_shared_t - one field there being 'state', planned to be used to mark the connection active/inactive.

Marking inactive is easy, when a command fails. Bring it back active is more complex, I thought of: - counting how many commands would have been sent when connection is inactive and if a threshold is reached, then try reconnect - keep the timestamp when connection became inactive and try to bring it active when a specified interval elapsed

Other suggestions/contributions are welcome -- the code is in the repo, so if anyone wants to jump in development, feel free to do it...

To your questions, doing serial or round robin writes in some databases and also using them for reads in the same fashion would require a replication at database layer.

I thought of cases such as: - master db servers for writes in parallel (e.g., location/presence in db only mode) - slave db servers (replicated from masters) for reads in round robin

Of course one could write to all servers and read from all servers, but if the traffic is very high, might be good to reuse some db layer features for scalability.

Cheers, Daniel

Cheers, Marius

...

Zbihlei Marius

Head of Linux Development Services Romania

1&1 Internet Development srl Tel KA: 754-9152 Str Mircea Eliade 18 Tel RO: +40-31-223-9152 Sect 1, Bucuresti mailto: marius.zbihlei@1and1.ro 71295, Romania

---

sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev