portalocker package

Submodules

Module contents

exception portalocker.AlreadyLocked(*args, fh=None, **kwargs)[source]

Bases: portalocker.exceptions.BaseLockException

class portalocker.BoundedSemaphore(maximum: int, name: str = 'bounded_semaphore', filename_pattern: str = '{name}.{number:02d}.lock', directory: str = '/tmp', timeout=5, check_interval=0.25)[source]

Bases: portalocker.utils.LockBase

Bounded semaphore to prevent too many parallel processes from running

It’s also possible to specify a timeout when acquiring the lock to wait for a resource to become available. This is very similar to threading.BoundedSemaphore but works across multiple processes and across multiple operating systems.

>>> semaphore = BoundedSemaphore(2, directory='')
>>> str(semaphore.get_filenames()[0])
'bounded_semaphore.00.lock'
>>> str(sorted(semaphore.get_random_filenames())[1])
'bounded_semaphore.01.lock'
acquire(timeout: Optional[float] = None, check_interval: Optional[float] = None, fail_when_locked: Optional[bool] = None) Optional[portalocker.utils.Lock][source]
get_filename(number) pathlib.Path[source]
get_filenames() Sequence[pathlib.Path][source]
get_random_filenames() Sequence[pathlib.Path][source]
lock: Optional[portalocker.utils.Lock]
release()[source]
try_lock(filenames: Sequence[Union[str, pathlib.Path]]) bool[source]
portalocker.LOCK_EX: portalocker.constants.LockFlags = LockFlags.EXCLUSIVE

Place an exclusive lock. Only one process may hold an exclusive lock for a given file at a given time.

portalocker.LOCK_NB: portalocker.constants.LockFlags = LockFlags.NON_BLOCKING

Acquire the lock in a non-blocking fashion.

portalocker.LOCK_SH: portalocker.constants.LockFlags = LockFlags.SHARED

Place a shared lock. More than one process may hold a shared lock for a given file at a given time.

portalocker.LOCK_UN: portalocker.constants.LockFlags = LockFlags.UNBLOCK

Remove an existing lock held by this process.

class portalocker.Lock(filename: Union[str, pathlib.Path], mode: str = 'a', timeout: float = 5, check_interval: float = 0.25, fail_when_locked: bool = False, flags: portalocker.constants.LockFlags = LockFlags.None, **file_open_kwargs)[source]

Bases: portalocker.utils.LockBase

Lock manager with build-in timeout

Parameters
  • filename – filename

  • mode – the open mode, ‘a’ or ‘ab’ should be used for writing

  • truncate – use truncate to emulate ‘w’ mode, None is disabled, 0 is truncate to 0 bytes

  • timeout – timeout when trying to acquire a lock

  • check_interval – check interval while waiting

  • fail_when_locked – after the initial lock failed, return an error or lock the file. This does not wait for the timeout.

  • **file_open_kwargs – The kwargs for the open(…) call

fail_when_locked is useful when multiple threads/processes can race when creating a file. If set to true than the system will wait till the lock was acquired and then return an AlreadyLocked exception.

Note that the file is opened first and locked later. So using ‘w’ as mode will result in truncate _BEFORE_ the lock is checked.

acquire(timeout: Optional[float] = None, check_interval: Optional[float] = None, fail_when_locked: Optional[bool] = None) IO[source]

Acquire the locked filehandle

check_interval: float

check interval while waiting for timeout

fail_when_locked: bool

skip the timeout and immediately fail if the initial lock fails

release()[source]

Releases the currently locked file handle

timeout: float

timeout when trying to acquire a lock

exception portalocker.LockException(*args, fh=None, **kwargs)[source]

Bases: portalocker.exceptions.BaseLockException

class portalocker.LockFlags(value)[source]

Bases: enum.IntFlag

An enumeration.

EXCLUSIVE = 2

exclusive lock

NON_BLOCKING = 4

non-blocking

SHARED = 1

shared lock

UNBLOCK = 8

unlock

class portalocker.RLock(filename, mode='a', timeout=5, check_interval=0.25, fail_when_locked=False, flags=LockFlags.None)[source]

Bases: portalocker.utils.Lock

A reentrant lock, functions in a similar way to threading.RLock in that it can be acquired multiple times. When the corresponding number of release() calls are made the lock will finally release the underlying file lock.

acquire(timeout: Optional[float] = None, check_interval: Optional[float] = None, fail_when_locked: Optional[bool] = None) IO[source]

Acquire the locked filehandle

check_interval: float

check interval while waiting for timeout

fail_when_locked: bool

skip the timeout and immediately fail if the initial lock fails

fh: Optional[IO]
filename: str
flags: portalocker.constants.LockFlags
mode: str
release()[source]

Releases the currently locked file handle

timeout: float

timeout when trying to acquire a lock

truncate: bool
class portalocker.RedisLock(channel: str, connection: Optional[redis.client.Redis] = None, timeout: Optional[float] = None, check_interval: Optional[float] = None, fail_when_locked: Optional[bool] = False, thread_sleep_time: float = 0.1, unavailable_timeout: float = 1, redis_kwargs: Optional[Dict] = None)[source]

Bases: portalocker.utils.LockBase

An extremely reliable Redis lock based on pubsub with a keep-alive thread

As opposed to most Redis locking systems based on key/value pairs, this locking method is based on the pubsub system. The big advantage is that if the connection gets killed due to network issues, crashing processes or otherwise, it will still immediately unlock instead of waiting for a lock timeout.

To make sure both sides of the lock know about the connection state it is recommended to set the health_check_interval when creating the redis connection..

Parameters
  • channel – the redis channel to use as locking key.

  • connection (or if you need to specify the redis) – an optional redis connection if you already have one

  • connection

  • timeout – timeout when trying to acquire a lock

  • check_interval – check interval while waiting

  • fail_when_locked – after the initial lock failed, return an error or lock the file. This does not wait for the timeout.

  • thread_sleep_time – sleep time between fetching messages from redis to prevent a busy/wait loop. In the case of lock conflicts this increases the time it takes to resolve the conflict. This should be smaller than the check_interval to be useful.

  • unavailable_timeout – If the conflicting lock is properly connected this should never exceed twice your redis latency. Note that this will increase the wait time possibly beyond your timeout and is always executed if a conflict arises.

  • redis_kwargs – The redis connection arguments if no connection is given. The DEFAULT_REDIS_KWARGS are used as default, if you want to override these you need to explicitly specify a value (e.g. health_check_interval=0)

DEFAULT_REDIS_KWARGS = {'health_check_interval': 10}
acquire(timeout: Optional[float] = None, check_interval: Optional[float] = None, fail_when_locked: Optional[bool] = None)[source]
channel: str
channel_handler(message)[source]
check_or_kill_lock(connection, timeout)[source]
property client_name
close_connection: bool
connection: Optional[redis.client.Redis]
get_connection() redis.client.Redis[source]
pubsub: Optional[redis.client.PubSub] = None
redis_kwargs: Dict[str, Any]
release()[source]
thread: Optional[portalocker.redis.PubSubWorkerThread]
timeout: float

timeout when trying to acquire a lock

portalocker.lock(file_: IO, flags: portalocker.constants.LockFlags)[source]

Lock a file. Note that this is an advisory lock on Linux/Unix systems

portalocker.open_atomic(filename: Union[str, pathlib.Path], binary: bool = True)[source]

Open a file for atomic writing. Instead of locking this method allows you to write the entire file and move it to the actual location. Note that this makes the assumption that a rename is atomic on your platform which is generally the case but not a guarantee.

http://docs.python.org/library/os.html#os.rename

>>> filename = 'test_file.txt'
>>> if os.path.exists(filename):
...     os.remove(filename)
>>> with open_atomic(filename) as fh:
...     written = fh.write(b'test')
>>> assert os.path.exists(filename)
>>> os.remove(filename)
>>> import pathlib
>>> path_filename = pathlib.Path('test_file.txt')
>>> with open_atomic(path_filename) as fh:
...     written = fh.write(b'test')
>>> assert path_filename.exists()
>>> path_filename.unlink()
portalocker.unlock(file_: IO)[source]

Unlock a file