portalocker package¶
Submodules¶
- portalocker.redis module
PubSubWorkerThread
RedisLock
RedisLock.DEFAULT_REDIS_KWARGS
RedisLock.acquire()
RedisLock.channel
RedisLock.channel_handler()
RedisLock.check_interval
RedisLock.check_or_kill_lock()
RedisLock.client_name
RedisLock.close_connection
RedisLock.connection
RedisLock.fail_when_locked
RedisLock.get_connection()
RedisLock.pubsub
RedisLock.redis_kwargs
RedisLock.release()
RedisLock.thread
RedisLock.timeout
- portalocker.constants module
- portalocker.exceptions module
- portalocker.portalocker module
- portalocker.utils module
Module contents¶
- exception portalocker.AlreadyLocked(*args: Any, fh: Optional[IO] = None, **kwargs: Any)[source]¶
Bases:
LockException
- class portalocker.BoundedSemaphore(maximum: int, name: str = 'bounded_semaphore', filename_pattern: str = '{name}.{number:02d}.lock', directory: str = '/tmp', timeout=5, check_interval=0.25)[source]¶
Bases:
LockBase
Bounded semaphore to prevent too many parallel processes from running
It’s also possible to specify a timeout when acquiring the lock to wait for a resource to become available. This is very similar to threading.BoundedSemaphore but works across multiple processes and across multiple operating systems.
>>> semaphore = BoundedSemaphore(2, directory='') >>> str(semaphore.get_filenames()[0]) 'bounded_semaphore.00.lock' >>> str(sorted(semaphore.get_random_filenames())[1]) 'bounded_semaphore.01.lock'
- portalocker.LOCK_EX: LockFlags = LockFlags.EXCLUSIVE¶
Place an exclusive lock. Only one process may hold an exclusive lock for a given file at a given time.
- portalocker.LOCK_NB: LockFlags = LockFlags.NON_BLOCKING¶
Acquire the lock in a non-blocking fashion.
- portalocker.LOCK_SH: LockFlags = LockFlags.SHARED¶
Place a shared lock. More than one process may hold a shared lock for a given file at a given time.
- class portalocker.Lock(filename: ~typing.Union[str, ~pathlib.Path], mode: str = 'a', timeout: ~typing.Optional[float] = None, check_interval: float = 0.25, fail_when_locked: bool = False, flags: ~portalocker.constants.LockFlags = LockFlags.None, **file_open_kwargs)[source]¶
Bases:
LockBase
Lock manager with built-in timeout
- Parameters
filename – filename
mode – the open mode, ‘a’ or ‘ab’ should be used for writing
truncate – use truncate to emulate ‘w’ mode, None is disabled, 0 is truncate to 0 bytes
timeout – timeout when trying to acquire a lock
check_interval – check interval while waiting
fail_when_locked – after the initial lock failed, return an error or lock the file. This does not wait for the timeout.
**file_open_kwargs – The kwargs for the open(…) call
fail_when_locked is useful when multiple threads/processes can race when creating a file. If set to true than the system will wait till the lock was acquired and then return an AlreadyLocked exception.
Note that the file is opened first and locked later. So using ‘w’ as mode will result in truncate _BEFORE_ the lock is checked.
- exception portalocker.LockException(*args: Any, fh: Optional[IO] = None, **kwargs: Any)[source]¶
Bases:
BaseLockException
- class portalocker.LockFlags(value)[source]¶
Bases:
IntFlag
An enumeration.
- EXCLUSIVE = 2¶
exclusive lock
- NON_BLOCKING = 4¶
non-blocking
- SHARED = 1¶
shared lock
- UNBLOCK = 8¶
unlock
- class portalocker.RLock(filename, mode='a', timeout=5, check_interval=0.25, fail_when_locked=False, flags=LockFlags.None)[source]¶
Bases:
Lock
A reentrant lock, functions in a similar way to threading.RLock in that it can be acquired multiple times. When the corresponding number of release() calls are made the lock will finally release the underlying file lock.
- class portalocker.RedisLock(channel: str, connection: Optional[Redis] = None, timeout: Optional[float] = None, check_interval: Optional[float] = None, fail_when_locked: Optional[bool] = False, thread_sleep_time: float = 0.1, unavailable_timeout: float = 1, redis_kwargs: Optional[Dict] = None)[source]¶
Bases:
LockBase
An extremely reliable Redis lock based on pubsub with a keep-alive thread
As opposed to most Redis locking systems based on key/value pairs, this locking method is based on the pubsub system. The big advantage is that if the connection gets killed due to network issues, crashing processes or otherwise, it will still immediately unlock instead of waiting for a lock timeout.
To make sure both sides of the lock know about the connection state it is recommended to set the health_check_interval when creating the redis connection..
- Parameters
channel – the redis channel to use as locking key.
connection (or if you need to specify the redis) – an optional redis connection if you already have one
connection –
timeout – timeout when trying to acquire a lock
check_interval – check interval while waiting
fail_when_locked – after the initial lock failed, return an error or lock the file. This does not wait for the timeout.
thread_sleep_time – sleep time between fetching messages from redis to prevent a busy/wait loop. In the case of lock conflicts this increases the time it takes to resolve the conflict. This should be smaller than the check_interval to be useful.
unavailable_timeout – If the conflicting lock is properly connected this should never exceed twice your redis latency. Note that this will increase the wait time possibly beyond your timeout and is always executed if a conflict arises.
redis_kwargs – The redis connection arguments if no connection is given. The DEFAULT_REDIS_KWARGS are used as default, if you want to override these you need to explicitly specify a value (e.g. health_check_interval=0)
- DEFAULT_REDIS_KWARGS = {'health_check_interval': 10}¶
- acquire(timeout: Optional[float] = None, check_interval: Optional[float] = None, fail_when_locked: Optional[bool] = None)[source]¶
- property client_name¶
- thread: Optional[PubSubWorkerThread]¶
- portalocker.lock(file_: IO, flags: LockFlags)[source]¶
Lock a file. Note that this is an advisory lock on Linux/Unix systems
- portalocker.open_atomic(filename: Union[str, Path], binary: bool = True) Iterator[IO] [source]¶
Open a file for atomic writing. Instead of locking this method allows you to write the entire file and move it to the actual location. Note that this makes the assumption that a rename is atomic on your platform which is generally the case but not a guarantee.
http://docs.python.org/library/os.html#os.rename
>>> filename = 'test_file.txt' >>> if os.path.exists(filename): ... os.remove(filename)
>>> with open_atomic(filename) as fh: ... written = fh.write(b'test') >>> assert os.path.exists(filename) >>> os.remove(filename)
>>> import pathlib >>> path_filename = pathlib.Path('test_file.txt')
>>> with open_atomic(path_filename) as fh: ... written = fh.write(b'test') >>> assert path_filename.exists() >>> path_filename.unlink()