Reflections on the Compatibility, Performance, and Scalability of Parallel Python
Today’s hardware is increasingly parallel, and to increase performance, applications must be able to use this parallelism. Hence, programming languages must provide the means for parallel execution. The language Python offers a multithreading, shared-memory model for concurrency. However, simultaneous execution of threads, i.e., parallel execution, is not a standard feature of current virtual machines (VM) for Python. Instead, the predominant Python VMs depend on a global interpreter lock, which serializes the execution.
In a parallel VM, replicating Python’s concurrency semantics is challenging. Today, there are three parallel VMs, which use one of two approaches to address the challenges: Jython, IronPython, and PyPy-STM. These VMs use two fundamentally different approaches to synchronize parallel execution under Python’s concurrency semantics: Jython and IronPython use fine-grained locking, and PyPy-STM uses software transactional memory (STM).
The two approaches result in different performance characteristics and levels of Python compatibility for these VMs. In this paper, we report on our experience with the three parallel VMs by comparing their compatibility, performance, and scalability. The comparison shows that fine-grained locking can yield better scalability than the STM approach. However, regarding the faithful reproduction of Python’s concurrency semantics and the absolute performance, the STM approach currently has the advantage.
Sun 20 Oct
|14:00 - 14:30|
|14:30 - 15:00|
Olivier FlückigerNortheastern University, Guido ChariCzech Technical University, Jan JecmenCzech Technical University, Ming-Ho YeeNortheastern University, Jakob HainNortheastern University, Jan VitekNortheastern UniversityLink to publication DOI Pre-print Media Attached
|15:00 - 15:30|
Python Programmers have GPUs too: Automatic Python Loop Parallelization with Staged Dependence AnalysisResearch Paper
Dejice JacobUniversity of Glasgow, Phil TrinderUniversity of Glasgow, Jeremy SingerUniversity of GlasgowLink to publication DOI Authorizer link