utils - Generic helper utilities

This module contains generic helper functions, not related to imaging specifically

class neural.utils.Beacon(app_name=None, instance_name=None, packet_path=None, poll_time=0.5)

Class to easily handle running multiple threads simultaneously. Communicates through a lockfile in an arbitrary file path, so communicating across different computers that have shared file systems is relatively easy (just choose a file path on the shared drive).

Options:

app_name:Arbitrary name of the script. Will use script filename if None instance_name:Arbitrary name of this instance (e.g., subject #) packet_path:Path to put the lock file in (defaults to the system temp directory) poll_time:How often (in seconds) to ping the lock file

Example of usage:

b = Beacon('my_analysis','subject_4')
if not b.exists():
    # there are no "my_analysis" scripts running "subject_4" right now
    with b.thread_safe():
        # lock this, so other scripts will fail when they run b.exists() on this subject
        # do the analysis here...

check_packet()

is there a valid packet (from another thread) for this app/instance?

exists()

Returns a bool of whether this analysis is already running somewhere else

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

thread_safe()

Use in a with statement to run code within a thread-safe context. When the with statement enters, this will create the lock file, and it will automatically stop and delete it when the with block finishes

class neural.utils.RunResult(output=None, return_code=None, output_filename=None)

result of calling run()

when used as a string, will try to reasonably return the filename of the primary output of the command (if known)

class neural.utils.ThreadSafe(beacon)

wrapper class to handle starting and stopping for the thread_safe() method of Beacon

neural.utils.archive_basename(filename)

returns the basename (name without extension) of a recognized archive file

neural.utils.compress(data, level=9)

return compressed, ASCII-encoded string

neural.utils.decompress(data)

return uncompressed string

neural.utils.factor(n)

return set of all prime factors for a number

neural.utils.find(file)

tries to find file using OS-specific searches and some guessing

neural.utils.flatten(nested_list)

converts a list-of-lists to a single flat list

neural.utils.hash(filename)

returns string of MD5 hash of given filename

neural.utils.hash_str(string)

returns string of MD5 hash of given string

neural.utils.is_archive(filename)

returns boolean of whether this filename looks like an archive

neural.utils.log(fname, msg)

generic logging function

neural.utils.numberize(string)

Turns a string into a number (int or float) if it’s only a number (ignoring spaces), otherwise returns the string. For example, "5 " becomes 5 and "2 ton" remains "2 ton"

neural.utils.random_string(length)

Returns random string of letters

neural.utils.run(command, products=None, working_directory='.', force_local=False, stderr=True, quiet=False)

wrapper to run external programs

Command:list containing command and parameters (formatted the same as subprocess; must contain only strings) Products:string or list of files that are the products of this command if all products exist, the command will not be run, and False returned Working_directory:  will chdir to this directory Force_local:when used with neural.scheduler, setting to True will disable all job distribution functions Stderr:forward stderr into the output True will combine stderr and stdout False will return stdout and let stderr print to the console None will return stdout and suppress stderr Quiet:False (default) will print friendly messages True will suppress everything but errors None will suppress all output

Returns result in form of RunResult

class neural.utils.run_in(working_directory, create=False)

temporarily changes into another directory

If the directory name you pass doesn’t exist, but matches the current directory you are in, it will silently ignore your silliness. Otherwise, it will raise an OSError. If the argument create is True, it will create the directory instead of raising an error.

Example::

with run_in(‘another_directory’):

do_some_stuff_there()

class neural.utils.run_in_tmp(inputs=[], products=[])

creates a temporary directory to run the code block in

class neural.utils.simple_timer

a simple way to time a single run of a function

Example::

with simple_timer():

do_stuff()

neural.utils.strip_rows(array, invalid=None)

takes a list of list``s and removes corresponding indices containing the invalid value (default ``None).

neural.utils.unarchive(filename, output_dir='.')

unpacks the given archive into output_dir

neural.utils.universal_read(fname)

Will open and read a file with universal line endings, trying to decode whatever format it’s in (e.g., utf8 or utf16)

neural.utils.which(program)

returns full path to program name or None if not found taken from: http://stackoverflow.com/questions/377017/test-if-executable-exists-in-python