IPC::Shareable(3pm) User Contributed Perl Documentation IPC::Shareable(3pm)
NAME
IPC::Shareable - Use shared memory backed variables across processes
SYNOPSIS
use IPC::Shareable qw(:lock);
my $href = IPC::Shareable->new(%options);
# ...or
tie SCALAR, 'IPC::Shareable', OPTIONS;
tie ARRAY, 'IPC::Shareable', OPTIONS;
tie HASH, 'IPC::Shareable', OPTIONS;
tied(VARIABLE)->lock;
tied(VARIABLE)->unlock;
tied(VARIABLE)->lock(LOCK_SH|LOCK_NB)
or print "Resource unavailable\n";
my $segment = tied(VARIABLE)->seg;
my $semaphore = tied(VARIABLE)->sem;
tied(VARIABLE)->remove;
IPC::Shareable::clean_up;
IPC::Shareable::clean_up_all;
IPC::Shareable::clean_up_protected;
# Ensure only one instance of a script can be run at any time
IPC::Shareable->singleton('UNIQUE SCRIPT LOCK STRING');
# Get the actual IPC::Shareable tied object
my $knot = tied(VARIABLE); # Dereference first if using a tied reference
DESCRIPTION
IPC::Shareable allows you to tie a variable to shared memory making it
easy to share the contents of that variable with other Perl processes
and scripts.
Scalars, arrays, hashes and even objects can be tied. The variable
being tied may contain arbitrarily complex data structures - including
references to arrays, hashes of hashes, etc.
The association between variables in distinct processes is provided by
GLUE (aka "key"). This is any arbitrary string or integer that serves
as a common identifier for data across process space. Hence the
statement:
tie my $scalar, 'IPC::Shareable', { key => 'GLUE STRING', create => 1 };
...in program one and the statement
tie my $variable, 'IPC::Shareable', { key => 'GLUE STRING' };
...in program two will create and bind $scalar the shared memory in
program one and bind it to $variable in program two.
There is no pre-set limit to the number of processes that can bind to
data; nor is there a pre-set limit to the complexity of the underlying
data of the tied variables. The amount of data that can be shared
within a single bound variable is limited by the system's maximum size
for a shared memory segment (the exact value is system-dependent).
The bound data structures are all linearized (using Raphael Manfredi's
Storable module or optionally JSON) before being slurped into shared
memory. Upon retrieval, the original format of the data structure is
recovered. Semaphore flags can be used for locking data between
competing processes.
OPTIONS
Options are specified by passing a reference to a hash as the third
argument to the "tie()" function that enchants a variable.
The following fields are recognized in the options hash:
key
key is the GLUE that is a direct reference to the shared memory segment
that's to be tied to the variable.
If this option is missing, we'll default to using "IPC_PRIVATE". This
default key will not allow sharing of the variable between processes.
Default: IPC_PRIVATE
create
create is used to control whether the process creates a new shared
memory segment or not. If create is set to a true value,
IPC::Shareable will create a new binding associated with GLUE as
needed. If create is false, IPC::Shareable will not attempt to create
a new shared memory segment associated with GLUE. In this case, a
shared memory segment associated with GLUE must already exist or we'll
"croak()".
Defult: false
exclusive
If exclusive field is set to a true value, we will "croak()" if the
data binding associated with GLUE already exists. If set to a false
value, calls to "tie()" will succeed even if a shared memory segment
associated with GLUE already exists.
See "graceful" for a silent, non-exception exit if a second process
attempts to obtain an in-use "exclusive" segment.
Default: false
graceful
If exclusive is set to a true value, we normally "croak()" if a second
process attempts to obtain the same shared memory segment. Set graceful
to true and we'll "exit" silently and gracefully. This option does
nothing if "exclusive" isn't set.
Useful for ensuring only a single process is running at a time.
Default: false
warn
When set to a true value, graceful will output a warning if there are
process collisions.
Default: false
mode
The mode argument is an octal number specifying the access permissions
when a new data binding is being created. These access permission are
the same as file access permissions in that 0666 is world readable,
0600 is readable only by the effective UID of the process creating the
shared variable, etc.
Default: 0666 (world read and writeable)
size
This field may be used to specify the size of the shared memory segment
allocated.
The maximum size we allow by default is ~1GB. See the "limit" option to
override this default.
Default: "IPC::Shareable::SHM_BUFSIZ()" (ie. 65536)
protected
If set, the "clean_up()" and "clean_up_all()" routines will not remove
the segments or semaphores related to the tied object.
Set this to a specific integer so we can pass the value to any child
objects created under the main one.
To clean up protected objects, call "(tied
%object)->clean_up_protected(integer)", where 'integer' is the value
you set the "protected" option to. You can call this cleanup routine in
the script you created the segment, or anywhere else, at any time.
Default: 0
limit
This field will allow you to set a segment size larger than the default
maximum which is 1,073,741,824 bytes (approximately 1 GB). If set, we
will "croak()" if a size specified is larger than the maximum. If it's
set to a false value, we'll "croak()" if you send in a size larger than
the total system RAM.
Default: true
destroy
If set to a true value, the shared memory segment underlying the data
binding will be removed when the process that initialized the shared
memory segment exits (gracefully)[1].
Only those memory segments that were created by the current process
will be removed.
Use this option with care. In particular you should not use this option
in a program that will fork after binding the data. On the other hand,
shared memory is a finite resource and should be released if it is not
needed.
NOTE: If the segment was created with its "protected" attribute set, it
will not be removed upon program completion, even if "destroy" is set.
Default: false
tidy
For long running processes, set this to a true value to clean up
unneeded segments from nested data structures. Comes with a slight
performance hit.
Default: false
serializer
By default, we use Storable as the data serializer when writing to or
reading from the shared memory segments we create. For cross-platform
and cross-language purposes, you can optionally use JSON for this task.
Send in either "json" or "storable" as the value to use the respective
serializer.
Default: storable
Default Option Values
Default values for options are:
key => IPC_PRIVATE, # 0
create => 0,
exclusive => 0,
mode => 0666,
size => IPC::Shareable::SHM_BUFSIZ(), # 65536
protected => 0,
limit => 1,
destroy => 0,
graceful => 0,
warn => 0,
tidy => 0,
serializer => 'storable',
METHODS
new
Instantiates and returns a reference to a hash backed by shared memory.
my $href = IPC::Shareable->new(key => "testing", create => 1);
$href=>{a} = 1;
# Call tied() on the dereferenced variable to access object methods
# and information
tied(%$href)->ipcs;
Parameters:
Hash, Optional: See the "OPTIONS" section for a list of all available
options. Most often, you'll want to send in the key and create
options.
It is possible to get a reference to an array or scalar as well. Simply
send in either "var = > 'ARRAY'" or "var => 'SCALAR'" to do so.
Return: A reference to a hash (or array or scalar) which is backed by
shared memory.
singleton($glue, $warn)
Class method that ensures that only a single instance of a script can
be run at any given time.
Parameters:
$glue
Mandatory, String: The key/glue that identifies the shared memory
segment.
$warn
Optional, Bool: Send in a true value to have subsequent processes throw
a warning that there's been a shared memory violation and that it will
exit.
Default: false
ipcs
Returns the number of instantiated shared memory segments that
currently exist on the system. This isn't precise; it simply does a "wc
-l" line count on your system's "ipcs -m" call. It is guaranteed though
to produce reliable results.
Return: Integer
lock($flags)
Obtains a lock on the shared memory. $flags specifies the type of lock
to acquire. If $flags is not specified, an exclusive read/write lock
is obtained. Acceptable values for $flags are the same as for the
"flock()" system call.
Returns "true" on success, and "undef" on error. For non-blocking calls
(see below), the method returns 0 if it would have blocked.
Obtain an exclusive lock like this:
tied(%var)->lock(LOCK_EX); # same as default
Only one process can hold an exclusive lock on the shared memory at a
given time.
Obtain a shared (read) lock:
tied(%var)->lock(LOCK_SH);
Multiple processes can hold a shared (read) lock at a given time. If a
process attempts to obtain an exclusive lock while one or more
processes hold shared locks, it will be blocked until they have all
finished.
Either of the locks may be specified as non-blocking:
tied(%var)->lock( LOCK_EX|LOCK_NB );
tied(%var)->lock( LOCK_SH|LOCK_NB );
A non-blocking lock request will return 0 if it would have had to wait
to obtain the lock.
Note that these locks are advisory (just like flock), meaning that all
cooperating processes must coordinate their accesses to shared memory
using these calls in order for locking to work. See the "flock()" call
for details.
Locks are inherited through forks, which means that two processes
actually can possess an exclusive lock at the same time. Don't do that.
The constants "LOCK_EX", "LOCK_SH", "LOCK_NB", and "LOCK_UN" are
available for import using any of the following export tags:
use IPC::Shareable qw(:lock);
use IPC::Shareable qw(:flock);
use IPC::Shareable qw(:all);
Or, just use the flock constants available in the Fcntl module.
See "LOCKING" for further details.
unlock
Removes a lock. Takes no parameters, returns "true" on success.
This is equivalent of calling "shlock(LOCK_UN)".
See "LOCKING" for further details.
seg
Called on either the tied variable or the tie object, returns the
shared memory segment object currently in use.
sem
Called on either the tied variable or the tie object, returns the
semaphore object related to the memory segment currently in use.
attributes
Retrieves the list of attributes that drive the IPC::Shareable object.
Parameters:
$attribute
Optional, String: The name of the attribute. If sent in, we'll return
the value of this specific attribute. Returns "undef" if the attribute
isn't found.
Attributes are the "OPTIONS" that were used to create the object.
Returns: A hash reference of all attributes if $attributes isn't sent
in, the value of the specific attribute if it is.
global_register
Returns a hash reference of hashes of all in-use shared memory segments
across all processes. The key is the memory segment ID, and the value
is the segment and semaphore objects.
process_register
Returns a hash reference of hashes of all in-use shared memory segments
created by the calling process. The key is the memory segment ID, and
the value is the segment and semaphore objects.
LOCKING
IPC::Shareable provides methods to implement application-level advisory
locking of the shared data structures. These methods are called
"lock()" and "unlock()". To use them you must first get the object
underlying the tied variable, either by saving the return value of the
original call to "tie()" or by using the built-in "tied()" function.
To lock and subsequently unlock a variable, do this:
my $knot = tie my %hash, 'IPC::Shareable', { %options };
$knot->lock;
$hash{a} = 'foo';
$knot->unlock;
or equivalently, if you've decided to throw away the return of "tie()":
tie my %hash, 'IPC::Shareable', { %options };
tied(%hash)->lock;
$hash{a} = 'foo';
tied(%hash)->unlock;
This will place an exclusive lock on the data of $scalar. You can also
get shared locks or attempt to get a lock without blocking.
IPC::Shareable makes the constants "LOCK_EX", "LOCK_SH", "LOCK_UN", and
"LOCK_NB" exportable to your address space with the export tags
":lock", ":flock", or ":all". The values should be the same as the
standard "flock" option arguments.
if (tied(%hash)->lock(LOCK_SH|LOCK_NB)){
print "The value is $hash{a}\n";
tied(%hash)->unlock;
} else {
print "Another process has an exclusive lock.\n";
}
If no argument is provided to "lock", it defaults to "LOCK_EX".
There are some pitfalls regarding locking and signals about which you
should make yourself aware; these are discussed in "NOTES".
Note that in the background, we perform lock optimization when reading
and writing to the shared storage even if the advisory locks aren't
being used.
Using the advisory locks can speed up processes that are doing several
writes/ reads at the same time.
DESTRUCTION
perl(1) will destroy the object underlying a tied variable when then
tied variable goes out of scope. Unfortunately for IPC::Shareable,
this may not be desirable: other processes may still need a handle on
the relevant shared memory segment.
IPC::Shareable therefore provides several options to control the timing
of removal of shared memory segments.
destroy Option
As described in "OPTIONS", specifying the destroy option when
"tie()"ing a variable coerces IPC::Shareable to remove the underlying
shared memory segment when the process calling "tie()" exits
gracefully.
NOTE: The destruction is handled in an "END" block. Only those memory
segments that are tied to the current process will be removed.
NOTE: If the segment was created with its "protected" attribute set, it
will not be removed in the "END" block, even if "destroy" is set.
remove
tied($var)->remove;
# or
$knot->remove;
Calling "remove()" on the object underlying a "tie()"d variable removes
the associated shared memory segments. The segment is removed
irrespective of whether it has the destroy option set or not and
irrespective of whether the calling process created the segment.
clean_up
IPC::Shareable->clean_up;
# or
tied($var)->clean_up;
# or
$knot->clean_up;
This is a class method that provokes IPC::Shareable to remove all
shared memory segments created by the process. Segments not created by
the calling process are not removed.
This method will not clean up segments created with the "protected"
option.
clean_up_all
IPC::Shareable->clean_up_all;
# or
tied($var)->clean_up_all;
# or
$knot->clean_up_all
This is a class method that provokes IPC::Shareable to remove all
shared memory segments encountered by the process. Segments are
removed even if they were not created by the calling process.
This method will not clean up segments created with the "protected"
option.
clean_up_protected($protect_key)
If a segment is created with the "protected" option, it, nor its
children will be removed during calls of "clean_up()" or
"clean_up_all()".
When setting "protected", you specified a lock key integer. When
calling this method, you must send that integer in as a parameter so we
know which segments to clean up.
my $protect_key = 93432;
IPC::Shareable->clean_up_protected($protect_key);
# or
tied($var)->clean_up_protected($protect_key;
# or
$knot->clean_up_protected($protect_key)
Parameters:
$protect_key
Mandatory, Integer: The integer protect key you assigned wit the
"protected" option
RETURN VALUES
Calls to "tie()" that try to implement IPC::Shareable will return an
instance of "IPC::Shareable" on success, and "undef" otherwise.
AUTHOR
Benjamin Sugars <bsugars@canoe.ca>
MAINTAINED BY
Steve Bertrand <steveb@cpan.org>
NOTES
Footnotes from the above sections
1. If the process has been smoked by an untrapped signal, the binding
will remain in shared memory. If you're cautious, you might try:
$SIG{INT} = \&catch_int;
sub catch_int {
die;
}
...
tie $variable, IPC::Shareable, { key => 'GLUE', create => 1, 'destroy' => 1 };
which will at least clean up after your user hits CTRL-C because
IPC::Shareable's END method will be called. Or, maybe you'd like
to leave the binding in shared memory, so subsequent process can
recover the data...
General Notes
o When using "lock()" to lock a variable, be careful to guard against
signals. Under normal circumstances, "IPC::Shareable"'s "END"
method unlocks any locked variables when the process exits.
However, if an untrapped signal is received while a process holds
an exclusive lock, "END" will not be called and the lock may be
maintained even though the process has exited. If this scares you,
you might be better off implementing your own locking methods.
One advantage of using "flock" on some known file instead of the
locking implemented with semaphores in "IPC::Shareable" is that
when a process dies, it automatically releases any locks. This
only happens with "IPC::Shareable" if the process dies gracefully.
The alternative is to attempt to account for every possible
calamitous ending for your process (robust signal handling in Perl
is a source of much debate, though it usually works just fine) or
to become familiar with your system's tools for removing shared
memory and semaphores. This concern should be balanced against the
significant performance improvements you can gain for larger data
structures by using the locking mechanism implemented in
IPC::Shareable.
o There is a program called "ipcs"(1/8) (and "ipcrm"(1/8)) that is
available on at least Solaris and Linux that might be useful for
cleaning moribund shared memory segments or semaphore sets produced
by bugs in either IPC::Shareable or applications using it.
Examples:
# List all semaphores and memory segments in use on the system
ipcs -a
# List all memory segments and semaphores along with each one's associated process ID
ipcs -ap
# List just the shared memory segments
ipcs -m
# List the details of an individual memory segment
ipcs -i 12345678
# Remove *all* semaphores and memory segments
ipcrm -a
o This version of IPC::Shareable does not understand the format of
shared memory segments created by versions prior to 0.60. If you
try to tie to such segments, you will get an error. The only work
around is to clear the shared memory segments and start with a
fresh set.
o Iterating over a hash causes a special optimization if you have not
obtained a lock (it is better to obtain a read (or write) lock
before iterating over a hash tied to IPC::Shareable, but we attempt
this optimization if you do not).
For tied hashes, the "fetch"/"thaw" operation is performed when the
first key is accessed. Subsequent key and and value accesses are
done without accessing shared memory. Doing an assignment to the
hash or fetching another value between key accesses causes the hash
to be replaced from shared memory. The state of the iterator in
this case is not defined by the Perl documentation. Caveat Emptor.
CREDITS
Thanks to all those with comments or bug fixes, especially
Maurice Aubrey <maurice@hevanet.com>
Stephane Bortzmeyer <bortzmeyer@pasteur.fr>
Doug MacEachern <dougm@telebusiness.co.nz>
Robert Emmery <roberte@netscape.com>
Mohammed J. Kabir <kabir@intevo.com>
Terry Ewing <terry@intevo.com>
Tim Fries <timf@dicecorp.com>
Joe Thomas <jthomas@women.com>
Paul Makepeace <Paul.Makepeace@realprogrammers.com>
Raphael Manfredi <Raphael_Manfredi@pobox.com>
Lee Lindley <Lee.Lindley@bigfoot.com>
Dave Rolsky <autarch@urth.org>
Steve Bertrand <steveb@cpan.org>
SEE ALSO
perltie, Storable, "shmget", "ipcs", "ipcrm" and other SysV IPC manual
pages.
perl v5.34.0 2022-10-15 IPC::Shareable(3pm)
Generated by dwww version 1.15 on Wed Jun 26 04:53:31 CEST 2024.