Welcome to portalocker’s documentation!¶
portalocker - Cross-platform locking library¶
Overview¶
Portalocker is a library to provide an easy API to file locking.
An important detail to note is that on Linux and Unix systems the locks are advisory by default. By specifying the -o mand option to the mount command it is possible to enable mandatory file locking on Linux. This is generally not recommended however. For more information about the subject:
The module is currently maintained by Rick van Hattem <Wolph@wol.ph>. The project resides at https://github.com/WoLpH/portalocker . Bugs and feature requests can be submitted there. Patches are also very welcome.
Redis Locks¶
This library now features a lock based on Redis which allows for locks across multiple threads, processes and even distributed locks across multiple computers.
It is an extremely reliable Redis lock that is based on pubsub.
As opposed to most Redis locking systems based on key/value pairs, this locking method is based on the pubsub system. The big advantage is that if the connection gets killed due to network issues, crashing processes or otherwise, it will still immediately unlock instead of waiting for a lock timeout.
First make sure you have everything installed correctly:
pip install "portalocker[redis]"
Usage is really easy:
import portalocker
lock = portalocker.RedisLock('some_lock_channel_name')
with lock:
print('do something here')
The API is essentially identical to the other Lock
classes so in addition
to the with
statement you can also use lock.acquire(...)
.
Python 2¶
Python 2 was supported in versions before Portalocker 2.0. If you are still using Python 2, you can run this to install:
pip install "portalocker<2"
Tips¶
On some networked filesystems it might be needed to force a os.fsync() before closing the file so it’s actually written before another client reads the file. Effectively this comes down to:
with portalocker.Lock('some_file', 'rb+', timeout=60) as fh:
# do what you need to do
...
# flush and sync to filesystem
fh.flush()
os.fsync(fh.fileno())
Links¶
- Documentation
- Bug reports
- Package homepage
- My blog
Examples¶
To make sure your cache generation scripts don’t race, use the Lock class:
>>> import portalocker
>>> with portalocker.Lock('somefile', timeout=1) as fh:
... print('writing some stuff to my cache...', file=fh)
To customize the opening and locking a manual approach is also possible:
>>> import portalocker
>>> file = open('somefile', 'r+')
>>> portalocker.lock(file, portalocker.LockFlags.EXCLUSIVE)
>>> file.seek(12)
>>> file.write('foo')
>>> file.close()
Explicitly unlocking is not needed in most cases but omitting it has been known to cause issues: https://github.com/AzureAD/microsoft-authentication-extensions-for-python/issues/42#issuecomment-601108266
If needed, it can be done through:
>>> portalocker.unlock(file)
Do note that your data might still be in a buffer so it is possible that your data is not available until you flush() or close().
To create a cross platform bounded semaphore across multiple processes you can use the BoundedSemaphore class which functions somewhat similar to threading.BoundedSemaphore:
>>> import portalocker
>>> n = 2
>>> timeout = 0.1
>>> semaphore_a = portalocker.BoundedSemaphore(n, timeout=timeout)
>>> semaphore_b = portalocker.BoundedSemaphore(n, timeout=timeout)
>>> semaphore_c = portalocker.BoundedSemaphore(n, timeout=timeout)
>>> semaphore_a.acquire()
<portalocker.utils.Lock object at ...>
>>> semaphore_b.acquire()
<portalocker.utils.Lock object at ...>
>>> semaphore_c.acquire()
Traceback (most recent call last):
...
portalocker.exceptions.AlreadyLocked
More examples can be found in the tests.
Versioning¶
This library follows Semantic Versioning.
Changelog¶
Every release has a git tag
with a commit message for the tag
explaining what was added and/or changed. The list of tags/releases
including the commit messages can be found here:
https://github.com/WoLpH/portalocker/releases
License¶
See the LICENSE file.
Contents:
- portalocker package
- Submodules
- Module contents
AlreadyLocked
BoundedSemaphore
LOCK_EX
LOCK_NB
LOCK_SH
LOCK_UN
Lock
LockException
LockFlags
RLock
RedisLock
RedisLock.DEFAULT_REDIS_KWARGS
RedisLock.acquire()
RedisLock.channel
RedisLock.channel_handler()
RedisLock.check_or_kill_lock()
RedisLock.client_name
RedisLock.close_connection
RedisLock.connection
RedisLock.get_connection()
RedisLock.pubsub
RedisLock.redis_kwargs
RedisLock.release()
RedisLock.thread
RedisLock.timeout
lock()
open_atomic()
unlock()
- tests package
- Module contents
LockResult
LockResult.__annotations__
LockResult.__dataclass_fields__
LockResult.__dataclass_params__
LockResult.__dict__
LockResult.__eq__()
LockResult.__ge__()
LockResult.__gt__()
LockResult.__hash__
LockResult.__init__()
LockResult.__le__()
LockResult.__lt__()
LockResult.__module__
LockResult.__repr__()
LockResult.__weakref__
LockResult.exception_class
LockResult.exception_message
LockResult.exception_repr
exclusive_lock()
lock()
shared_lock()
shared_lock_fail()
test_acquire_release()
test_blocking_timeout()
test_class()
test_exceptions()
test_exclusive_processes()
test_exlusive()
test_lock_fileno()
test_nonblocking()
test_release_unacquired()
test_rlock_acquire_release()
test_rlock_acquire_release_count()
test_shared()
test_shared_processes()
test_simple()
test_truncate()
test_utils_base()
test_with_timeout()
test_without_fail()
test_without_timeout()
test_combined()
test_temporary_file_lock()
- Module contents
- License