We've been seeing crashes in libbfd when we attempt to generate
a stack trace from multiple threads. It turns out that libbfd
is NOT thread-safe. It can cache the bfd structure and give it to
multiple threads without protecting itself. To get around this,
we've added a global mutex around the bfd functions and also have
refactored the use of those functions to be more efficient and
to provide more information about inlined functions.
Also added a few more tests to test_pbx.c. One just calls
ast_assert() and the other calls ast_log_backtrace(). Neither are
run by default.
WARNING: This change necessitated changing the return value of
ast_bt_get_symbols() from an array of strings to a VECTOR of
strings. However, the use of this function outside Asterisk is not
likely.
ASTERISK-28140
Change-Id: I79d02862ddaa2423a0809caa4b3b85c128131621
Default logging was not setup correctly when there was no logger.conf.
This resulted in many expected log messages not actually getting out to
the console.
Change-Id: I542e61c03b2f630ff5327f9de5641d776c6fa70c
All log messages go to a queue serviced by a single thread
which does all the IO. This setting controls how big that
queue can get (and therefore how much memory is allocated)
before new messages are discarded. The default is 1000.
Should something go bezerk and log tons of messages in a tight
loop, this will prevent memory escalation.
When the limit is reached, a WARNING is logged to that effect
and messages are discarded until the queue is empty again. At
that time another WARNING will be logged with the count of
discarded messages. There's no "low water mark" for this queue
because the logger thread empties the entire queue and processes it
in 1 batch before going back and waiting on the queue again.
Implementing a low water mark would mean additional locking as
the thread processes each message and it's not worth it.
A "test" was added to test_logger.c but since the outcome is
non-deterministic, it's really just a cli command, not a unit
test.
Change-Id: Ib4520c95e1ca5325dbf584c7989ce391649836d1
In all non-pbx modules, AST_MODULE_LOAD_FAILURE has been changed
to AST_MODULE_LOAD_DECLINE. This prevents asterisk from exiting
if a module can't be loaded. If the user wishes to retain the
FAILURE behavior for a specific module, they can use the "require"
or "preload-require" keyword in modules.conf.
A new API was added to logger: ast_is_logger_initialized(). This
allows asterisk.c/check_init() to print to the error log once the
logger subsystem is ready instead of just to stdout. If something
does fail before the logger is initialized, we now print to stderr
instead of stdout.
Change-Id: I5f4b50623d9b5a6cb7c5624a8c5c1274c13b2b25
When logger.conf is missing or invalid we should be printing notices,
warnings and errors to the console. The logmask was incorrectly
calculated.
Change-Id: Ibaa9465a8682854bc1a5e9ba07079bea1bfb6bb3
The stringfields refactor to allow adding stringfields to the end of a
structure (f6f4cf459f) exposed some
incomplete cleanup code by some stringfield users.
The most noticeable leaker is the logging system where there is a leak for
every log message generated.
ASTERISK-26078 #close
Reported by: Etienne Lessard
Patches:
jira_asterisk_26078_v13.patch (license #5621) patch uploaded
by Richard Mudgett
Change-Id: If6a08b31336b492c3de6f9dfd07c447f8d5a8782
During refactoring of this support the addition of
the PID to messages was removed. This change adds it
back in.
ASTERISK-25538 #close
Change-Id: Ie2d43b0652e59b7ac319a7dba94501540d70ba36
The fix to ASTERISK-25407 introduced the usage of LOG_MAKEPRI. However
this macro is broken in older glibc (< 2.17); it would left-shift the
facility a second time, causing the resultant priority to become
invalid.
The syslog manpage mentions nothing about LOG_MAKEPRI and suggests this:
The priority argument is formed by ORing the facility and the level
values [...].
ASTERISK-25510 #close
Reported by: Michael Newton
Change-Id: Ia89debe7fac5ad090c7ef595c0707f31bb1e3d03
The null terminator of the tail struct member was not being allocated
when no logger.conf config file is installed.
ASTERISK-25714 #close
Reported by: Badalian Vyacheslav
Change-Id: I45770fdd08af39506a3bc33ba279c4f16e047a30
There was a problem observed where the "logger add channel" CLI command
would allow for a channel with the same name to be added multiple times.
This would result in each message being written out to the same file
multiple times.
The problem was due to the difference in how logger channel filenames
are stored versus the format they are allowed to be presented when they
are added. For instance, if adding the logger channel "foo" through the
CLI, the result would be a logger channel with the file name
/var/log/asterisk/foo being stored. So when trying to add another "foo"
channel, "foo" would not match "/var/log/asterisk/foo" so we'd happily
add the duplicate channel.
The fix presented here is to introduce two new methods in the logger
code:
* make_filename(): given a logger channel name, this creates the
filename for that logger channel.
* find_logchannel(): given a logger channel name, this calls
make_filename() and then traverses the list of logchannels in order
to find a match.
This change has made use of make_filename() and find_logchannel()
throughout to more consistently behave.
ASTERISK-25305 #close
Reported by Mark Michelson
Change-Id: I892d52954d6007d8bc453c3cbdd9235dec9c4a36
Currently, Asterisk will log to the last configured syslog
channel in logger.conf. This is due to the fact that the
final call to openlog() supersedes all of the previous calls.
This commit removes the call to openlog() and passes the
facility to ast_log_vsyslog(), along with utilizing the
LOG_MAKEPRI macro to ensure that the message is routed to
the correct facility and with the correct priority.
ASTERISK-25407 #close
Reported by: Elazar Broad
Tested by: Elazar Broad
Change-Id: Ie2a2416bc00cce1b04e99ef40917c2011953ddd2
An http request can be sent to get the existing Asterisk logs.
The command "curl -v -u user:pass -X GET 'http://localhost:8088
/ari/asterisk/logging'" can be run in the terminal to access the
newly implemented functionality.
* Retrieve all existing log channels
ASTERISK-25252
Change-Id: I7bb08b93e3b938c991f3f56cc5d188654768a808
An http request can be sent to create a log channel
in Asterisk.
The command "curl -v -u user:pass -X POST
'http://localhost:088/ari/asterisk/logging/mylog?
configuration=notice,warning'" can be run in the terminal
to access the newly implemented functionality for ARI.
* Ability to create log channels using ARI
ASTERISK-25252
Change-Id: I9a20e5c75716dfbb6b62fd3474faf55be20bd782
An http request can be sent to delete a log channel
in Asterisk.
The command "curl -v -u user:pass -X DELETE 'http://localhost:8088
/ari/asterisk/logging/mylog'" can be run in the terminal
to access the newly implemented functionally for ARI.
* Able to delete log channels using ARI
ASTERISK-25252
Change-Id: Id6eeb54ebcc511595f0418d586ff55914bc3aae6
An http request can be sent to rotate a specified log channel.
If the channel does not exist, an error response will be
returned.
The command "curl -v -u user:pass -X PUT 'http://localhost:8088
/ari/asterisk/logging/logChannelName/rotate'" can be run in the
terminal to access this new functionality.
* Added the ability to rotate log files through ARI
ASTERISK-25252
Change-Id: Iaefa21cbbc1b29effb33004ee3d89c977e76ab01
Reset options to default values before reloading config. This ensures
that if a setting is removed or commented out of the configuration file
it is unset on reload.
ASTERISK-25112 #close
Reported by: Corey Farrell
Change-Id: Id24bb1fb0885c2c14cf8bd6f69a0c2ee7cd6c5bd
This introduces a new logger routine ast_log_safe. This routine should be
used for all error messages in code that can be run as a result of ast_log.
ast_log_safe does nothing if run recursively. All error logging in
astobj2.c, strings.c and utils.h have been switched to ast_log_safe.
This required adding support for raw threadstorage. This provides direct
access to the void* pointer in threadstorage. In ast_log_safe, NULL is used
to signify that this thread is not already running ast_log_safe, (void*)1 when
it is already running. This was done since it's critical that ast_log_safe
do nothing that could log during recursion checking.
ASTERISK-24155 #close
Reported by: Timo Teräs
Review: https://reviewboard.asterisk.org/r/4502/
........
Merged revisions 433522 from http://svn.asterisk.org/svn/asterisk/branches/11
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@433523 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The QUEUESTART log entry has historically acted like a fully booted event
for the queue_log file. When the QUEUESTART entry was posted to the log
was broken by the change made by ASTERISK-15863.
* Made post the QUEUESTART queue_log entry when Asterisk fully boots.
This restores the intent of that log entry and happens after realtime has
had a chance to load.
AST-1444 #close
Reported by: Denis Martinez
Review: https://reviewboard.asterisk.org/r/4282/
........
Merged revisions 430009 from http://svn.asterisk.org/svn/asterisk/branches/11
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@430010 65c4cc65-6c06-0410-ace0-fbb531ad65f3
If a reference count goes negative, instead of
just logging that fact, be more helpful with a
backtrace and an assert that will DO_CRASH.
This patch also removes the duplicate ao2_bt()
function and cleans up extraneous usage of the
ast_log_backtrace() call.
Review: https://reviewboard.asterisk.org/r/3765/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@418963 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch addresses some aesthetic issues in Asterisk. These are all just
minor tweaks to improve the look of the CLI when used in a variety of
settings. Specifically:
* A number of chatty verbose messages were removed or demoted to DEBUG
messages. Verbose messages with a verbosity level of 5 or higher were -
if kept as verbose messages - demoted to level 4. Several messages
that were emitted at verbose level 3 were demoted to 4, as announcement
of dialplan applications being executed occur at level 3 (and so the
effects of those applications should generally be less).
* Some verbose messages that only appear when their respective 'debug'
options are enabled were bumped up to always be displayed.
* Prefix/timestamping of verbose messages were moved to the verboser
handlers. This was done to prevent duplication of prefixes when the
timestamp option (-T) is used with the CLI.
* Verbose magic is removed from messages before being emitted to
non-verboser handlers. This prevents the magic in multi-line verbose
messages (such as SIP debug traces or the output of DumpChan) from
being written to files.
* _Slightly_ better support for the "light background" option (-W) was
added. This includes using ast_term_quit in the output of XML
documentation help, as well as changing the "Asterisk Ready" prompt to
bright green on the default background (which stands a better chance of
being displayed properly than bright white).
Review: https://reviewboard.asterisk.org/r/3547/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@414798 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The per console verbose level feature as previously implemented caused a
large performance penalty. The fix required some minor incompatibilities
if the new rasterisk is used to connect to an earlier version. If the new
rasterisk connects to an older Asterisk version then the root console
verbose level is always affected by the "core set verbose" command of the
remote console even though it may appear to only affect the current
console. If an older version of rasterisk connects to the new version
then the "core set verbose" command will have no effect.
* Fixed the verbose performance by not generating a verbose message if
nothing is going to use it and then filtered any generated verbose
messages before actually sending them to the remote consoles.
* Split the "core set debug" and "core set verbose" CLI commands to remove
the per module verbose support that cannot work with the per console
verbose level.
* Added a silent option to the "core set verbose" command.
* Fixed "core set debug off" tab completion.
* Made "core show settings" list the current console verbosity in addition
to the root console verbosity.
* Changed the default verbose level of the 'verbose' setting in the
logger.conf [logfiles] section. The default is now to once again follow
the current root console level. As a result, using the AMI Command action
with "core set verbose" could again set the root console verbose level and
affect the verbose level logged.
(closes issue AST-1252)
Reported by: Guenther Kelleter
Review: https://reviewboard.asterisk.org/r/3114/
........
Merged revisions 405431 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 405432 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@405436 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The remote console continued to have issues with its output. In this case CLI
command output would either not show up (if verbose level = 0) or would contain
verbose prefixes (if verbose level > 0) once log messages were sent to the
remote console. The fix now now adds verbose prefix data to all new lines
contained in a verbose log string.
(closes issue ASTERISK-22450)
Reported by: David Brillert
(closes issue AST-1193)
Reported by: Guenther Kelleter
Review: https://reviewboard.asterisk.org/r/2825/
........
Merged revisions 399267 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 399268 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@399269 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Refactored cases where a combination of ast_verbose/options_verbose were
present. Also in general tried to eliminate, in as many places as possible,
where the options_verbose global variable was being used. Refactored the way
local and remote consoles handle verbose message logging in an attempt to
solve the various discrepancies that sometimes would show between the two.
(closes issue AST-1193)
Reported by: Guenther Kelleter
Review: https://reviewboard.asterisk.org/r/2798/
........
Merged revisions 397948 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 397958 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397959 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Review https://reviewboard.asterisk.org/r/2580/ tried to fix the mismatch
in memory pools but had a math error determining the buffer size and
didn't address other similar memory pool mismatches.
* Effectively reverted the previous patch to go in the same direction as
trunk for the returned memory pool of ast_bt_get_symbols().
* Fixed memory leak in ast_bt_get_symbols() when BETTER_BACKTRACES is
defined.
* Fixed some formatting in ast_bt_get_symbols().
* Fixed sig_pri.c freeing memory allocated by libpri when MALLOC_DEBUG is
enabled.
* Fixed __dump_backtrace() freeing memory from ast_bt_get_symbols() when
MALLOC_DEBUG is enabled.
* Moved __dump_backtrace() because of compile issues with the utils
directory.
(closes issue ASTERISK-22221)
Reported by: Matt Jordan
Review: https://reviewboard.asterisk.org/r/2778/
........
Merged revisions 397525 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........
Merged revisions 397528 from http://svn.asterisk.org/svn/asterisk/branches/11
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397570 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch allows astmm to access the backtrace generation code in Asterisk.
When memory is allocated, a backtrace is created and stored with the memory
region that tracks the allocation. If a memory corruption is detected, the
backtrace is printed to the astmm log. The backtrace will make use of the
BETTER_BACKTRACES build option if available.
As a result, this patch moves the backtrace generation code into its own file
and uses the non-wrapped versions of the C library memory allocation routines.
This allows the memory allocation code to safely use the backtrace generation
routines without infinitely recursing.
Review: https://reviewboard.asterisk.org/r/2567
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@391012 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When Asterisk shuts down and shuts down the loggin gsubsystem, any
messages currently in flight will not get logged. This patch prevents the
loop writing messages from breaking out prematurely, such that all of the
messages are logged.
(closes issue ASTERISK-21716)
Reported by: Corey Farrell
patches:
logger-process-all-messages.patch uploaded by Corey Farrell (license 5909)
........
Merged revisions 389676 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........
Merged revisions 389677 from http://svn.asterisk.org/svn/asterisk/branches/11
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@389680 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The CALL-ID (ie [C-00000074]) is missing when logging to syslog. This was just
an oversight when this feature was added.
* Add CALL-IDs when using syslog
(closes issue ASTERISK-21430)
Reported by: Nikola Ciprich
Tested by: Nikola Ciprich, Michael L. Young
Patches:
asterisk-21430-syslog-callid_trunk.diff by Michael L. Young (license 5026)
Review: https://reviewboard.asterisk.org/r/2526/
........
Merged revisions 388605 from http://svn.asterisk.org/svn/asterisk/branches/11
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@388617 65c4cc65-6c06-0410-ace0-fbb531ad65f3
While autoservice is running and servicing a channel the callid is being stored
and removed in the thread's local storage for each iteration of the thread loop.
If debug was set to a sufficient level the log file would be spammed with callid
thread local storage debug messages.
Added a new function that checks to see if the callid to be stored is different
than what is already contained (if anything). If it is different then
store/replace and log, otherwise just leave as is. Also made it so all logging
of debug messages pertaining to the callid thread storage outputs only when
TEST_FRAMEWORK is defined.
(issue ASTERISK-21014)
(closes issue ASTERISK-21014)
Report by: Rusty Newton
Review: https://reviewboard.asterisk.org/r/2324/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@381557 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The core module related to coloring terminal output was old and needed
some love. The main thing here was an attempt to get rid of the
obscene number of stack-local buffers that were allocated for no other
reason than to colorize some output. Instead, this uses a simple trick
to allocate several buffers within threadlocal storage, then
automatically rotates between them, so that you can make multiple calls
to the colorization routine within one function and not need to
allocate multiple buffers.
Review: https://reviewboard.asterisk.org/r/2241/
Patches:
bug.patch uploaded by Tilghman Lesher
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@381448 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Similar to the patch that moved the fork earlier in the startup sequence to
prevent mutex errors in the recursive mutex surrounding the read/write thread
registration lock, this patch re-initializes the logmsgs mutex. Part of the
start up sequence before forking the process into the background includes
reading asterisk.conf; this has to occur prior to the call to daemon in order
to read startup parameters. When reading in a conf file, log statements can
be generated. Since this can't be avoided, the mutex instead is
re-initialized to ensure a reset of any thread tracking information.
This patch also includes some additional debugging to catch errors when
locking or unlocking the recursive mutex that surrounds locks when the
DEBUG_THREADS build option is enabled. DO_CRASH or THREAD_CRASH will
cause an abort() if a mutex error is detected.
(issue ASTERISK-19463)
Reported by: mjordan
Tesetd by: mjordan
........
Merged revisions 376586 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........
Merged revisions 376587 from http://svn.asterisk.org/svn/asterisk/branches/10
........
Merged revisions 376588 from http://svn.asterisk.org/svn/asterisk/branches/11
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@376589 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Update and extend the configuration_file group and enable linking. Commit other cleanups from multi-version Doxygen testing. Update title that was left behind many years ago.
(issue ASTERISK-20259)
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@375182 65c4cc65-6c06-0410-ace0-fbb531ad65f3
With this option in use, it may be necessary to regulate your log files
externally.
(closes issue ASTERISK-20189)
Reported by: Jaco Kroon
Patches:
asterisk-logger-norotate-trunk.patch uploaded by Jaco Kroon (license 5671)
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@372976 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The awk script parses out the first instance of the DOCUMENTATION tag that it
finds within a file. If a file did not previously have a DOCUMENTATION tag
but received one due to it having an AMI event, then the XML fragment
associated with the AMI event was erroneously placed in the resulting XML
file. Without the python scripts, these XML fragments will not validate.
This patch adds DOCUMENTATION tags at the top of those files that did
not previously have them to prevent the awk script from pulling AMI event
documentation.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@369910 65c4cc65-6c06-0410-ace0-fbb531ad65f3
