Merge pull request #1 from NarrativeScience/init-code-commit

Initial code commit

Author: CelestineKao

.circleci/config.yml

@@ -16,7 +16,7 @@ commands:
 jobs:
   test:
     docker:
-      - image: circleci/python:3.7
+      - image: circleci/python:3.6
     steps:
       - ghpr/build-prospective-branch
       - install-deps
@@ -34,7 +34,7 @@ jobs:
           when: on_fail
   publish:
     docker:
-      - image: circleci/python:3.7
+      - image: circleci/python:3.6
     steps:
       - checkout
       - run: pip install flit

README.md

@@ -1,34 +1,67 @@
-# my-package-name
+# async-task-qeue
 
-[![](https://img.shields.io/pypi/v/my-package-name.svg)](https://pypi.org/pypi/my-package-name/) [![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
+[![](https://img.shields.io/pypi/v/sfn-workflow-client.svg)](https://pypi.org/pypi/async-task-queue/) [![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
 
-<!-- Short description of the package -->
+In-memory FIFO queue for concurrent task execution. Used to execute tasks concurrently with optional control (via semaphore) over the max number of tasks running at the same time.
 
 Features:
 
-- <!-- list of features -->
+- Queue processing summary logging
+- Introspection of failed, retried, and succeeded tasks
+- Task retries (optional)
+- Task execution timeout (optional)
+- Queue processing with semaphore (optional)
+- Batch size control (optional)
 
-Table of Contents:
+TOC:
 
 - [Installation](#installation)
 - [Guide](#guide)
 - [Development](#development)
 
 ## Installation
 
-my-package-name requires Python 3.6 or above.
+async-task-queue requires Python 3.6 or above.
 
 ```bash
-pip install my-package-name
+pip install async-task-queue
 ```
 
 ## Guide
 
-<!-- Subsections explaining how to use the package -->
+```python
+import logging
+from async_task_queue import AsyncTask, AsyncTaskQueue
+
+# Initialize a logger
+logger = logging.getLogger("foo")
+
+# Initialize an AsyncTaskQueue where:
+#   - At most 5 tasks are running concurrently
+#   - Number of tasks executing concurrently should be limited by a semaphore
+#   - Failed tasks should be retried (default behavior)
+#   - Executing the tasks queued should timeout and be cancelled after 5 minutes
+task_queue = AsyncTaskQueue(
+    logger,
+    use_semaphore=True,
+    batch_size=5,
+    execution_timeout=300
+)
+
+# Add async tasks to the queue
+task_queue.enqueue(
+    [
+        AsyncTask(some_coroutine, arg) for arg in some_args
+    ]
+)
+
+# Start processing the queue
+await task_queue.execute()
+```
 
 ## Development
 
-To develop my-package-name, install dependencies and enable the pre-commit hook:
+To develop async-task-queue, install dependencies and enable the pre-commit hook:
 
 ```bash
 pip install pre-commit tox

pyproject.toml

@@ -3,23 +3,20 @@ requires = ["flit_core >=2,<3"]
 build-backend = "flit_core.buildapi"
 
 [tool.flit.metadata]
-module = "my_package_name"
-author = "Jerry Seinfeld"
-author-email = "jseinfeld@narrativescience.com"
-home-page = "https://github.com/NarrativeScience/my-package-name"
+module = "async_task_queue"
+author = "Celestine Kao"
+author-email = "ckao@narrativescience.com"
+home-page = "https://github.com/NarrativeScience/async-task-queue"
 license = "BSD-3-Clause"
 description-file = "README.md"
-requires = [
-    # Requirements here, e.g. "backoff>=1.8.0,<2",
-]
 classifiers = [
     "Intended Audience :: Developers",
     "License :: OSI Approved :: BSD License",
     "Programming Language :: Python :: 3",
     "Topic :: Software Development :: Libraries :: Python Modules",
 ]
 requires-python = ">=3.6,<4"
-keywords = "comma,separated,keywords"
+keywords = "asyncio, async queue"
 
 [tool.flit.sdist]
 include = ["LICENSE.md"]

setup.cfg

@@ -10,7 +10,8 @@ force_sort_within_sections=true
 include_trailing_comma=true
 known_standard_library=typing
 known_first_party=
-    my_package_name
+    async_task_queue
+    test_utils
 line_length=88
 multi_line_output=3
 no_lines_before=LOCALFOLDER

src/async_task_queue/__init__.py

@@ -0,0 +1,4 @@
+"""In-memory FIFO queue for concurrent task execution"""
+from .async_task_queue import AsyncTask, AsyncTaskQueue
+
+__version__ = "0.1.0"