Topic names now follow: <subsystem>:<functionality>[/<object>]
This ensures that they are all unique, and also provides better
insight in to what each topic is for.
Subscriber ids now also use the main topic name they are
subscribed to and an incrementing integer as their identifier to
make it easier to understand what the subscription is primarily
responsible for.
Both the CLI commands for listing topic and subscription statistics
now sort to make it a bit easier to see what is going on.
Subscriptions will now show all topics that they are receiving messages
from, not just the main topic they were subscribed to.
ASTERISK-28335
Change-Id: I484e971a38c3640f2bd156282e532eed84bf220d
This change adds the ability for subscriptions to indicate
which message types they are interested in accepting. By
doing so the filtering is done before being dispatched
to the subscriber, reducing the amount of work that has
to be done.
This is optional and if a subscriber does not add
message types they wish to accept and set the subscription
to selective filtering the previous behavior is preserved
and they receive all messages.
There is also the ability to explicitly force the reception
of all messages for cases such as AMI or ARI where a large
number of messages are expected that are then generically
converted into a different format.
ASTERISK-28103
Change-Id: I99bee23895baa0a117985d51683f7963b77aa190
Somehow stasis_cache_pattern got out of sync between 13 and master
and it was causing duplicate channel message issues in 13 when
related to a specific endpoint. I.E. from statsd,
'endpoints.PJSIP.1174.channels 0|g' was being emitted twice.
Backporting stasis_cache_pattern from master to 13 solved
the issue and running the unit and testsuite tests confirmed
that no new ones were created.
ASTERISK-25317 #close
Change-Id: Ia8707462f62d15eed14541c37f332a7bbbceb548
When an endpoint is created, its messages are forwarded to both the tech
endpoint topic and the all endpoints topic. This is done so that various
parties interested in endpoint messages can subscribe to just the tech
endpoint and receive all messages associated with that particular technology,
as opposed to subscribing to the all endpoints topic. Unfortunately, when the
tech endpoint is created, it also forwards all of its messages to the all
topic. This results in duplicate messages whenever an endpoint publishes its
messages.
This patch resolves the duplicate message issue by creating a new function
for Stasis caching topics, stasis_cp_sink_create. In most respects, this acts
as a normal caching topic, save that it no longer forwards messages it receives
to the all endpoints topic. This allows it to act as an aggregation "sink",
while preserving the necessary caching behaviour.
ASTERISK-25137 #close
Reported-by: Vitezslav Novy
ASTERISK-25116 #close
Reported-by: George Joseph <george.joseph@fairview5.com>
Tested-by: George Joseph <george.joseph@fairview5.com>
Change-Id: Ie47784adfb973ab0063e59fc18f390d7dd26d17b
When an endpoint was created, it's messages were being forwarded to
both the tech endpoint topic and the all endpoints topic. Since
the tech topic was also forwarded to all, this was resulting in
duplicate messages whenever an endpoint published. This patch
causes the endpoint to only forward to the tech topic and lets
the tech topic forward to all.
To accomplish this, the existing stasis_cp_single_create function
(which both creates and forwards) was cloned and split into 2
functions, one that creates the topic and one that sets up the
forwarding. This allows endpoint_internal_create to create
the topic from the endpoint_all cache without forwarding it there,
then allows it to do the forward to the tech's topic.
ASTERISK-25137 #close
Reported-by: Vitezslav Novy
ASTERISK-25116 #close
Reported-by: George Joseph <george.joseph@fairview5.com>
Tested-by: George Joseph <george.joseph@fairview5.com>
Change-Id: I26d7d4926a0861748fd3bdffe316b75b549a801c
........
r399887 | dlee | 2013-09-26 10:41:47 -0500 (Thu, 26 Sep 2013) | 1 line
Minor performance bump by not allocate manager variable struct if we don't need it
........
r400138 | dlee | 2013-09-30 10:24:00 -0500 (Mon, 30 Sep 2013) | 23 lines
Stasis performance improvements
This patch addresses several performance problems that were found in
the initial performance testing of Asterisk 12.
The Stasis dispatch object was allocated as an AO2 object, even though
it has a very confined lifecycle. This was replaced with a straight
ast_malloc().
The Stasis message router was spending an inordinate amount of time
searching hash tables. In this case, most of our routers had 6 or
fewer routes in them to begin with. This was replaced with an array
that's searched linearly for the route.
We more heavily rely on AO2 objects in Asterisk 12, and the memset()
in ao2_ref() actually became noticeable on the profile. This was
#ifdef'ed to only run when AO2_DEBUG was enabled.
After being misled by an erroneous comment in taskprocessor.c during
profiling, the wrong comment was removed.
Review: https://reviewboard.asterisk.org/r/2873/
........
r400178 | dlee | 2013-09-30 13:26:27 -0500 (Mon, 30 Sep 2013) | 24 lines
Taskprocessor optimization; switch Stasis to use taskprocessors
This patch optimizes taskprocessor to use a semaphore for signaling,
which the OS can do a better job at managing contention and waiting
that we can with a mutex and condition.
The taskprocessor execution was also slightly optimized to reduce the
number of locks taken.
The only observable difference in the taskprocessor implementation is
that when the final reference to the taskprocessor goes away, it will
execute all tasks to completion instead of discarding the unexecuted
tasks.
For systems where unnamed semaphores are not supported, a really
simple semaphore implementation is provided. (Which gives identical
performance as the original taskprocessor implementation).
The way we ended up implementing Stasis caused the threadpool to be a
burden instead of a boost to performance. This was switched to just
use taskprocessors directly for subscriptions.
Review: https://reviewboard.asterisk.org/r/2881/
........
r400180 | dlee | 2013-09-30 13:39:34 -0500 (Mon, 30 Sep 2013) | 28 lines
Optimize how Stasis forwards are dispatched
This patch optimizes how forwards are dispatched in Stasis.
Originally, forwards were dispatched as subscriptions that are invoked
on the publishing thread. This did not account for the vast number of
forwards we would end up having in the system, and the amount of work it
would take to walk though the forward subscriptions.
This patch modifies Stasis so that rather than walking the tree of
forwards on every dispatch, when forwards and subscriptions are changed,
the subscriber list for every topic in the tree is changed.
This has a couple of benefits. First, this reduces the workload of
dispatching messages. It also reduces contention when dispatching to
different topics that happen to forward to the same aggregation topic
(as happens with all of the channel, bridge and endpoint topics).
Since forwards are no longer subscriptions, the bulk of this patch is
simply changing stasis_subscription objects to stasis_forward objects
(which, admittedly, I should have done in the first place.)
Since this required me to yet again put in a growing array, I finally
abstracted that out into a set of ast_vector macros in
asterisk/vector.h.
Review: https://reviewboard.asterisk.org/r/2883/
........
r400181 | dlee | 2013-09-30 13:48:57 -0500 (Mon, 30 Sep 2013) | 28 lines
Remove dispatch object allocation from Stasis publishing
While looking for areas for performance improvement, I realized that an
unused feature in Stasis was negatively impacting performance.
When a message is sent to a subscriber, a dispatch object is allocated
for the dispatch, containing the topic the message was published to, the
subscriber the message is being sent to, and the message itself.
The topic is actually unused by any subscriber in Asterisk today. And
the subscriber is associated with the taskprocessor the message is being
dispatched to.
First, this patch removes the unused topic parameter from Stasis
subscription callbacks.
Second, this patch introduces the concept of taskprocessor local data,
data that may be set on a taskprocessor and provided along with the data
pointer when a task is pushed using the ast_taskprocessor_push_local()
call. This allows the task to have both data specific to that
taskprocessor, in addition to data specific to that invocation.
With those two changes, the dispatch object can be removed completely,
and the message is simply refcounted and sent directly to the
taskprocessor.
Review: https://reviewboard.asterisk.org/r/2884/
........
Merged revisions 399887,400138,400178,400180-400181 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@400186 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Change r395954 reordered some stasis object destruction, which should
have been fine. Unfortunately, it caused some hard to reproduce issues
related to objects being accessed after they had been destroyed. The
patch in r396329 fixed the destruction order problem; this patch
addresses the underlying issue. A few other stasis-related fixes were
also added.
* Add ref-bumps around areas where objects may get transitively
destroyed. (For example, where we lock a topic, unref a subscription,
which unrefs the topic, which explodes the topic when we try to
unlock it.)
* Wrote an extensive doxygen page about Stasis implementation,
relationships between objects, lifecycles of objects, how the
refcounting works, etc. Many other comments were added, corrected, or
cleaned up.
* Added an assert to the topic dtor to catch extra ref decrements.
* Fixed type used after destruction errors for graceful shutdown in
stasis_channels.c.
* I added two unit tests in an attempt to catch destruction order
issues. Since the underlying cause is a race condition, though, the
tests rarely failed even when the code was wrong.
* Fixed a leak in stasis_cache_pattern.c.
(closes issue ASTERISK-22243)
Review: https://reviewboard.asterisk.org/r/2746/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396842 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The Stasis changes in r395954 had an unanticipated side effect: messages
published directly to an _all topic does not get forwarded to the
corresponding caching topic.
This patch fixes that by changing how caching topics forward messages,
and how the caching pattern forwards are setup.
For the caching pattern, the all_topic is forwarded to the
all_topic_cached. This forwards messages published directly to the
all_topic to all_topic_cached.
In order to avoid duplicate messages on all_topic_cached, caching topics
were changed to no longer forward uncached messages. Subscribers to an
individual caching topic should only expect to receive cache updates,
and subscription change messages. Since individual caching topics are
new, this shouldn't be a problem.
There are a few minor changes to the pre-cache split behavior.
* For topics changed to use the caching pattern, the all_topic_cached
will forward snapshots in addition to cache updates. Since
subscribers by design ignore unexpected messages, this should be
fine.
* Caching topics that don't use the caching pattern no longer forward
non-cache updates. This makes no difference for the current caching
topics.
* mwi_topic_cached, channel_by_name_topic and
presence_state_topic_cached have no subscribers
* device_state_topic_cached's only subscriber only processes cache
udpates
(issue ASTERISK-22243)
Review: https://reviewboard.asterisk.org/r/2738
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396329 65c4cc65-6c06-0410-ace0-fbb531ad65f3
In working with res_stasis, I discovered a significant limitation to
the current structure of stasis_caching_topics: you cannot subscribe
to cache updates for a single channel/bridge/endpoint/etc.
To address this, this patch splits the cache away from the
stasis_caching_topic, making it a first class object. The stasis_cache
object is shared amongst individual stasis_caching_topics that are
created per channel/endpoint/etc. These are still forwarded to global
whatever_all_cached topics, so their use from most of the code does
not change.
In making these changes, I noticed that we frequently used a similar
pattern for bridges, endpoints and channels:
single_topic ----------------> all_topic
^
|
single_topic_cached ----+----> all_topic_cached
|
+----> cache
This pattern was extracted as the 'Stasis Caching Pattern', defined in
stasis_caching_pattern.h. This avoids a lot of duplicate code between
the different domain objects.
Since the cache is now disassociated from its upstream caching topics,
this also necessitated a change to how the 'guaranteed' flag worked
for retrieving from a cache. The code for handling the caching
guarantee was extracted into a 'stasis_topic_wait' function, which
works for any stasis_topic.
(closes issue ASTERISK-22002)
Review: https://reviewboard.asterisk.org/r/2672/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395954 65c4cc65-6c06-0410-ace0-fbb531ad65f3
