Files
asterisk/res/res_pjsip/config_global.c
George Joseph c948ce9651 sorcery/res_pjsip: Refactor for realtime performance
There were a number of places in the res_pjsip stack that were getting
all endpoints or all aors, and then filtering them locally.

A good example is pjsip_options which, on startup, retrieves all
endpoints, then the aors for those endpoints, then tests the aors to see
if the qualify_frequency is > 0.  One issue was that it never did
anything with the endpoints other than retrieve the aors so we probably
could have skipped a step and just retrieved all aors. But nevermind.

This worked reasonably well with local config files but with a realtime
backend and thousands of objects, this was a nightmare.  The issue
really boiled down to the fact that while realtime supports predicates
that are passed to the database engine, the non-realtime sorcery
backends didn't.

They do now.

The realtime engines have a scheme for doing simple comparisons. They
take in an ast_variable (or list) for matching, and the name of each
variable can contain an operator.  For instance, a name of
"qualify_frequency >" and a value of "0" would create a SQL predicate
that looks like "where qualify_frequency > '0'".  If there's no operator
after the name, the engines add an '=' so a simple name of
"qualify_frequency" and a value of "10" would return exact matches.

The non-realtime backends decide whether to include an object in a
result set by calling ast_sorcery_changeset_create on every object in
the internal container.  However, ast_sorcery_changeset_create only does
exact string matches though so a name of "qualify_frequency >" and a
value of "0" returns nothing because the literal "qualify_frequency >"
doesn't match any name in the objset set.

So, the real task was to create a generic string matcher that can take a
left value, operator and a right value and perform the match. To that
end, strings.c has a new ast_strings_match(left, operator, right)
function.  Left and right are the strings to operate on and the operator
can be a string containing any of the following: = (or NULL or ""), !=,
>, >=, <, <=, like or regex.  If the operator is like or regex, the
right string should be a %-pattern or a regex expression.  If both left
and right can be converted to float, then a numeric comparison is
performed, otherwise a string comparison is performed.

To use this new function on ast_variables, 2 new functions were added to
config.c.  One that compares 2 ast_variables, and one that compares 2
ast_variable lists.  The former is useful when you want to compare 2
ast_variables that happen to be in a list but don't want to traverse the
list.  The latter will traverse the right list and return true if all
the variables in it match the left list.

Now, the backends' fields_cmp functions call ast_variable_lists_match
instead of ast_sorcery_changeset_create and they can now process the
same syntax as the realtime engines.  The realtime backend just passes
the variable list unaltered to the engine.  The only gotcha is that
there's no common realtime engine support for regex so that's been noted
in the api docs for ast_sorcery_retrieve_by_fields.

Only one more change to sorcery was done...  A new config flag
"allow_unqualified_fetch" was added to reg_sorcery_realtime.
"no": ignore fetches if no predicate fields were supplied.
"error": same as no but emit an error. (good for testing)
"yes": allow (the default);
"warn": allow but emit a warning. (good for testing)

Now on to res_pjsip...

pjsip_options was modified to retrieve aors with qualify_frequency > 0
rather than all endpoints then all aors.  Not only was this a big
improvement in realtime retrieval but even for config files there's an
improvement because we're not going through endpoints anymore.

res_pjsip_mwi was modified to retieve only endpoints with something in
the mailboxes field instead of all endpoints then testing mailboxes.

res_pjsip_registrar_expire was completely refactored.  It was retrieving
all contacts then setting up scheduler entries to check for expiration.
Now, it's a single thread (like keepalive) that periodically retrieves
only contacts whose expiration time is < now and deletes them.  A new
contact_expiration_check_interval was added to global with a default of
30 seconds.

Ross Beer reports that with this patch, his Asterisk startup time dropped
from around an hour to under 30 seconds.

There are still objects that can't be filtered at the database like
identifies, transports, and registrations.  These are not going to be
anywhere near as numerous as endpoints, aors, auths, contacts however.

Back to allow_unqualified_fetch.  If this is set to yes and you have a
very large number of objects in the database, the pjsip CLI commands
will attempt to retrive ALL of them if not qualified with a LIKE.
Worse, if you type "pjsip show endpoint <tab>" guess what's going to
happen? :)  Having a cache helps but all the objects will have to be
retrieved at least once to fill the cache.  Setting
allow_unqualified_fetch=no prevents the mass retrieve and should be used
on endpoints, auths, aors, and contacts.  It should NOT be used for
identifies, registrations and transports since these MUST be
retrieved in bulk.

Example sorcery.conf:

[res_pjsip]
endpoint=config,pjsip.conf,criteria=type=endpoint
endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error

ASTERISK-25826 #close
Reported-by: Ross Beer
Tested-by: Ross Beer

Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
2016-03-27 22:43:27 -05:00

363 lines
9.7 KiB
C

/*
* Asterisk -- An open source telephony toolkit.
*
* Copyright (C) 2013, Digium, Inc.
*
* Mark Michelson <mmichelson@digium.com>
*
* See http://www.asterisk.org for more information about
* the Asterisk project. Please do not directly contact
* any of the maintainers of this project for assistance;
* the project provides a web site, mailing lists and IRC
* channels for your use.
*
* This program is free software, distributed under the terms of
* the GNU General Public License Version 2. See the LICENSE file
* at the top of the source tree.
*/
#include "asterisk.h"
#include <pjsip.h>
#include <pjlib.h>
#include "asterisk/res_pjsip.h"
#include "include/res_pjsip_private.h"
#include "asterisk/sorcery.h"
#include "asterisk/ast_version.h"
#include "asterisk/res_pjsip_cli.h"
#define DEFAULT_MAX_FORWARDS 70
#define DEFAULT_KEEPALIVE_INTERVAL 0
#define DEFAULT_USERAGENT_PREFIX "Asterisk PBX"
#define DEFAULT_OUTBOUND_ENDPOINT "default_outbound_endpoint"
#define DEFAULT_DEBUG "no"
#define DEFAULT_ENDPOINT_IDENTIFIER_ORDER "ip,username,anonymous"
#define DEFAULT_MAX_INITIAL_QUALIFY_TIME 0
#define DEFAULT_FROM_USER "asterisk"
#define DEFAULT_REGCONTEXT ""
#define DEFAULT_CONTACT_EXPIRATION_CHECK_INTERVAL 30
static char default_useragent[256];
struct global_config {
SORCERY_OBJECT(details);
AST_DECLARE_STRING_FIELDS(
AST_STRING_FIELD(useragent);
AST_STRING_FIELD(regcontext);
AST_STRING_FIELD(default_outbound_endpoint);
/*! Debug logging yes|no|host */
AST_STRING_FIELD(debug);
/*! Order by which endpoint identifiers are checked (comma separated list) */
AST_STRING_FIELD(endpoint_identifier_order);
/*! User name to place in From header if there is no better option */
AST_STRING_FIELD(default_from_user);
);
/* Value to put in Max-Forwards header */
unsigned int max_forwards;
/* The interval at which to send keep alive messages to active connection-oriented transports */
unsigned int keep_alive_interval;
/* The maximum time for all contacts to be qualified at startup */
unsigned int max_initial_qualify_time;
/* The interval at which to check for expired contacts */
unsigned int contact_expiration_check_interval;
};
static void global_destructor(void *obj)
{
struct global_config *cfg = obj;
ast_string_field_free_memory(cfg);
}
static void *global_alloc(const char *name)
{
struct global_config *cfg;
cfg = ast_sorcery_generic_alloc(sizeof(*cfg), global_destructor);
if (!cfg || ast_string_field_init(cfg, 100)) {
ao2_cleanup(cfg);
return NULL;
}
return cfg;
}
static int global_apply(const struct ast_sorcery *sorcery, void *obj)
{
struct global_config *cfg = obj;
char max_forwards[10];
snprintf(max_forwards, sizeof(max_forwards), "%u", cfg->max_forwards);
ast_sip_add_global_request_header("Max-Forwards", max_forwards, 1);
ast_sip_add_global_request_header("User-Agent", cfg->useragent, 1);
ast_sip_add_global_response_header("Server", cfg->useragent, 1);
return 0;
}
static struct global_config *get_global_cfg(void)
{
struct global_config *cfg;
struct ao2_container *globals;
globals = ast_sorcery_retrieve_by_fields(ast_sip_get_sorcery(), "global",
AST_RETRIEVE_FLAG_MULTIPLE | AST_RETRIEVE_FLAG_ALL, NULL);
if (!globals) {
return NULL;
}
cfg = ao2_find(globals, NULL, 0);
ao2_ref(globals, -1);
return cfg;
}
char *ast_sip_global_default_outbound_endpoint(void)
{
char *str;
struct global_config *cfg;
cfg = get_global_cfg();
if (!cfg) {
return ast_strdup(DEFAULT_OUTBOUND_ENDPOINT);
}
str = ast_strdup(cfg->default_outbound_endpoint);
ao2_ref(cfg, -1);
return str;
}
char *ast_sip_get_debug(void)
{
char *res;
struct global_config *cfg;
cfg = get_global_cfg();
if (!cfg) {
return ast_strdup(DEFAULT_DEBUG);
}
res = ast_strdup(cfg->debug);
ao2_ref(cfg, -1);
return res;
}
char *ast_sip_get_regcontext(void)
{
char *res;
struct global_config *cfg;
cfg = get_global_cfg();
if (!cfg) {
return ast_strdup(DEFAULT_REGCONTEXT);
}
res = ast_strdup(cfg->regcontext);
ao2_ref(cfg, -1);
return res;
}
char *ast_sip_get_endpoint_identifier_order(void)
{
char *res;
struct global_config *cfg;
cfg = get_global_cfg();
if (!cfg) {
return ast_strdup(DEFAULT_ENDPOINT_IDENTIFIER_ORDER);
}
res = ast_strdup(cfg->endpoint_identifier_order);
ao2_ref(cfg, -1);
return res;
}
unsigned int ast_sip_get_keep_alive_interval(void)
{
unsigned int interval;
struct global_config *cfg;
cfg = get_global_cfg();
if (!cfg) {
return DEFAULT_KEEPALIVE_INTERVAL;
}
interval = cfg->keep_alive_interval;
ao2_ref(cfg, -1);
return interval;
}
unsigned int ast_sip_get_contact_expiration_check_interval(void)
{
unsigned int interval;
struct global_config *cfg;
cfg = get_global_cfg();
if (!cfg) {
return DEFAULT_CONTACT_EXPIRATION_CHECK_INTERVAL;
}
interval = cfg->contact_expiration_check_interval;
ao2_ref(cfg, -1);
return interval;
}
unsigned int ast_sip_get_max_initial_qualify_time(void)
{
unsigned int time;
struct global_config *cfg;
cfg = get_global_cfg();
if (!cfg) {
return DEFAULT_MAX_INITIAL_QUALIFY_TIME;
}
time = cfg->max_initial_qualify_time;
ao2_ref(cfg, -1);
return time;
}
void ast_sip_get_default_from_user(char *from_user, size_t size)
{
struct global_config *cfg;
cfg = get_global_cfg();
if (!cfg) {
ast_copy_string(from_user, DEFAULT_FROM_USER, size);
} else {
ast_copy_string(from_user, cfg->default_from_user, size);
ao2_ref(cfg, -1);
}
}
/*!
* \internal
* \brief Observer to set default global object if none exist.
*
* \param name Module name owning the sorcery instance.
* \param sorcery Instance being observed.
* \param object_type Name of object being observed.
* \param reloaded Non-zero if the object is being reloaded.
*
* \return Nothing
*/
static void global_loaded_observer(const char *name, const struct ast_sorcery *sorcery, const char *object_type, int reloaded)
{
struct ao2_container *globals;
struct global_config *cfg;
if (strcmp(object_type, "global")) {
/* Not interested */
return;
}
globals = ast_sorcery_retrieve_by_fields(sorcery, "global",
AST_RETRIEVE_FLAG_MULTIPLE | AST_RETRIEVE_FLAG_ALL, NULL);
if (globals) {
int count;
count = ao2_container_count(globals);
ao2_ref(globals, -1);
if (1 < count) {
ast_log(LOG_ERROR,
"At most one pjsip.conf type=global object can be defined. You have %d defined.\n",
count);
return;
}
if (count) {
return;
}
}
ast_debug(1, "No pjsip.conf type=global object exists so applying defaults.\n");
cfg = ast_sorcery_alloc(sorcery, "global", NULL);
if (!cfg) {
return;
}
global_apply(sorcery, cfg);
ao2_ref(cfg, -1);
}
static const struct ast_sorcery_instance_observer observer_callbacks_global = {
.object_type_loaded = global_loaded_observer,
};
int sip_cli_print_global(struct ast_sip_cli_context *context)
{
struct global_config *cfg = get_global_cfg();
if (!cfg) {
cfg = ast_sorcery_alloc(ast_sip_get_sorcery(), "global", NULL);
if (!cfg) {
return -1;
}
}
ast_str_append(&context->output_buffer, 0, "\nGlobal Settings:\n\n");
ast_sip_cli_print_sorcery_objectset(cfg, context, 0);
ao2_ref(cfg, -1);
return 0;
}
int ast_sip_destroy_sorcery_global(void)
{
struct ast_sorcery *sorcery = ast_sip_get_sorcery();
ast_sorcery_instance_observer_remove(sorcery, &observer_callbacks_global);
return 0;
}
int ast_sip_initialize_sorcery_global(void)
{
struct ast_sorcery *sorcery = ast_sip_get_sorcery();
snprintf(default_useragent, sizeof(default_useragent), "%s %s",
DEFAULT_USERAGENT_PREFIX, ast_get_version());
ast_sorcery_apply_default(sorcery, "global", "config", "pjsip.conf,criteria=type=global");
if (ast_sorcery_object_register(sorcery, "global", global_alloc, NULL, global_apply)) {
return -1;
}
ast_sorcery_object_field_register(sorcery, "global", "type", "", OPT_NOOP_T, 0, 0);
ast_sorcery_object_field_register(sorcery, "global", "max_forwards",
__stringify(DEFAULT_MAX_FORWARDS),
OPT_UINT_T, 0, FLDSET(struct global_config, max_forwards));
ast_sorcery_object_field_register(sorcery, "global", "user_agent", default_useragent,
OPT_STRINGFIELD_T, 0, STRFLDSET(struct global_config, useragent));
ast_sorcery_object_field_register(sorcery, "global", "default_outbound_endpoint",
DEFAULT_OUTBOUND_ENDPOINT,
OPT_STRINGFIELD_T, 0, STRFLDSET(struct global_config, default_outbound_endpoint));
ast_sorcery_object_field_register(sorcery, "global", "debug", DEFAULT_DEBUG,
OPT_STRINGFIELD_T, 0, STRFLDSET(struct global_config, debug));
ast_sorcery_object_field_register(sorcery, "global", "endpoint_identifier_order",
DEFAULT_ENDPOINT_IDENTIFIER_ORDER,
OPT_STRINGFIELD_T, 0, STRFLDSET(struct global_config, endpoint_identifier_order));
ast_sorcery_object_field_register(sorcery, "global", "keep_alive_interval",
__stringify(DEFAULT_KEEPALIVE_INTERVAL),
OPT_UINT_T, 0, FLDSET(struct global_config, keep_alive_interval));
ast_sorcery_object_field_register(sorcery, "global", "max_initial_qualify_time",
__stringify(DEFAULT_MAX_INITIAL_QUALIFY_TIME),
OPT_UINT_T, 0, FLDSET(struct global_config, max_initial_qualify_time));
ast_sorcery_object_field_register(sorcery, "global", "default_from_user", DEFAULT_FROM_USER,
OPT_STRINGFIELD_T, 0, STRFLDSET(struct global_config, default_from_user));
ast_sorcery_object_field_register(sorcery, "global", "regcontext", DEFAULT_REGCONTEXT,
OPT_STRINGFIELD_T, 0, STRFLDSET(struct global_config, regcontext));
ast_sorcery_object_field_register(sorcery, "global", "contact_expiration_check_interval",
__stringify(DEFAULT_CONTACT_EXPIRATION_CHECK_INTERVAL),
OPT_UINT_T, 0, FLDSET(struct global_config, contact_expiration_check_interval));
if (ast_sorcery_instance_observer_add(sorcery, &observer_callbacks_global)) {
return -1;
}
return 0;
}