Implement vanilla RNN

.github/pull_request_template.md

@@ -0,0 +1,23 @@
+Changelog
+---------
+
+### Added
+
+### Changed
+
+### Deprecated
+
+### Removed
+
+### Fixed
+
+### Security
+
+
+Checklist
+---------
+
+- [ ] Test
+- [ ] Self-review
+- [ ] Documentation
+- [ ] Version Bumped Manually*

.github/workflows/ci-cd.yml

@@ -0,0 +1,41 @@
+# Copyright Jiaqi Liu
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+---
+name: CI/CD
+
+"on":
+  pull_request:
+  push:
+    branches:
+      - master
+
+jobs:
+  release:
+    name: Publish Lamassu To PyPI
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.10"
+      - name: Package up SDK
+        if: github.ref == 'refs/heads/master'
+        run: python setup.py sdist
+      - name: Publish a Python distribution to PyPI
+        if: github.ref == 'refs/heads/master'
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          user: __token__
+          password: ${{ secrets.PYPI_API_TOKEN }}

.gitignore

@@ -0,0 +1,2 @@
+.idea/
+.DS_Store

.readthedocs.yaml

@@ -0,0 +1,26 @@
+# Copyright Jiaqi Liu
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+version: 2
+
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"
+
+sphinx:
+  configuration: docs/source/conf.py
+
+python:
+  install:
+  - requirements: docs/source/requirements.txt

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -0,0 +1,48 @@
+<div align="center">
+
+<img src="lamassu-logo.png" width="30%"/>
+
+</div>
+
+![Python Version][Python Version Badge]
+[![GitHub Workflow Status][GitHub Workflow Status badge]][GitHub Workflow Status URL]
+[![Read the Docs][Read the Docs badge]][Read the Docs URL]
+[![PyPI][PyPI project badge]][PyPI project url]
+[![Apache License badge]][Apache License URL]
+
+Lamassu
+=======
+
+Lamassu is a project that empowers individual to agnostically run machine learning algorithms to produce ad-hoc NLP
+features.
+
+Documentation
+-------------
+
+[**Lamassu is in beta development phase for the moment**](https://lamassu.readthedocs.io/en/latest/)
+
+License
+-------
+
+The use and distribution terms for Lamassu are covered by the
+[Apache License, Version 2.0][Apache License, Version 2.0].
+
+<div align="center">
+    <a href="https://opensource.org/licenses">
+        <img align="center" width="50%" alt="License Illustration" src="https://github.com/QubitPi/QubitPi/blob/master/img/apache-2.png?raw=true">
+    </a>
+</div>
+
+[Apache License badge]: https://img.shields.io/badge/Apache%202.0-F25910.svg?style=for-the-badge&logo=Apache&logoColor=white
+[Apache License URL]: https://www.apache.org/licenses/LICENSE-2.0
+[Apache License, Version 2.0]: http://www.apache.org/licenses/LICENSE-2.0.html
+
+[GitHub Workflow Status badge]: https://img.shields.io/github/actions/workflow/status/QubitPi/lamassu/ci-cd.yml?logo=github&style=for-the-badge
+[GitHub Workflow Status URL]: https://github.com/QubitPi/lamassu/actions/workflows/ci-cd.yml
+
+[Python Version Badge]: https://img.shields.io/badge/Python-3.10-brightgreen?style=for-the-badge&logo=python&logoColor=white
+[PyPI project badge]: https://img.shields.io/pypi/v/lamassu?logo=pypi&logoColor=white&style=for-the-badge
+[PyPI project url]: https://pypi.org/project/lamassu/
+
+[Read the Docs badge]: https://img.shields.io/readthedocs/lamassu?style=for-the-badge&logo=readthedocs&logoColor=white&label=Read%20the%20Docs&labelColor=8CA1AF
+[Read the Docs URL]: https://lamassu.readthedocs.io/en/latest/

docs/Makefile

@@ -0,0 +1,34 @@
+# Copyright Jiaqi Liu
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.http://sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/source/conf.py

@@ -0,0 +1,96 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+sys.path.insert(0, os.path.abspath('../../'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = 'lamassu'
+copyright = '2023, Jiaqi Liu'
+author = 'Jiaqi Liu'
+
+# The full version, including alpha/beta/rc tags
+release = '0.1.0'
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.autodoc',
+    'hoverxref.extension',
+    'notfound.extension',
+    'sphinx.ext.coverage',
+    'sphinx.ext.intersphinx',
+    'sphinx.ext.viewcode',
+    "sphinx.ext.graphviz",
+    "pyan.sphinx"
+]
+
+# add graphviz options
+graphviz_output_format = "svg"
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = []
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'sphinx_rtd_theme'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+intersphinx_mapping = {
+    'attrs': ('https://www.attrs.org/en/stable/', None),
+    'coverage': ('https://coverage.readthedocs.io/en/stable', None),
+    'cryptography': ('https://cryptography.io/en/latest/', None),
+    'cssselect': ('https://cssselect.readthedocs.io/en/latest', None),
+    'itemloaders': ('https://itemloaders.readthedocs.io/en/latest/', None),
+    'pytest': ('https://docs.pytest.org/en/latest', None),
+    'python': ('https://docs.python.org/3', None),
+    'sphinx': ('https://www.sphinx-doc.org/en/master', None),
+    'tox': ('https://tox.wiki/en/latest/', None),
+    'twisted': ('https://docs.twisted.org/en/stable/', None),
+    'twistedapi': ('https://docs.twisted.org/en/stable/api/', None),
+    'w3lib': ('https://w3lib.readthedocs.io/en/latest', None),
+}
+intersphinx_disabled_reftypes = []
+
+hoverxref_auto_ref = True
+hoverxref_role_types = {
+    "class": "tooltip",
+    "command": "tooltip",
+    "confval": "tooltip",
+    "hoverxref": "tooltip",
+    "mod": "tooltip",
+    "ref": "tooltip",
+    "reqmeta": "tooltip",
+    "setting": "tooltip",
+    "signal": "tooltip",
+}
+hoverxref_roles = ['command', 'reqmeta', 'setting', 'signal']

docs/source/faq.rst

@@ -0,0 +1,79 @@
+.. _faq:
+
+==========================
+Frequently Asked Questions
+==========================
+
+
+Python Sphinx Autodoc Is Not Rendering on readthedocs
+=====================================================
+
+The project's dependencies are not specified on RTD, but instead have installed the dependencies locally. We can verify
+this in the `build log <https://readthedocs.org/api/v2/build/17857935.txt>`_. Visit the project's Builds, click a build,
+and click "view raw"::
+
+    WARNING: autodoc: failed to import module 'rnn' from module 'lamassu'; the following exception was raised:
+    No module named 'matplotlib'
+
+To remedy the situation, we must specify that the project's dependencies to be installed. See
+`Specifying Dependencies <https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html>`_.
+
+
+Generate Sphinx Documentation Locally
+=====================================
+
+This site is auto-generated using `Sphinx <https://www.sphinx-doc.org/en/master/>`_ with the following command in venv::
+
+    cd /path/to/lamassu/
+    python3 -m venv venv
+    source venv/bin/activate
+    pip3 install .
+    pip3 install -r docs/source/requirements.txt
+    sphinx-build -a -b html docs/source/ /path/to/html/output/dir
+    deactivate
+
+.. NOTE::
+   The command above works for Linux/UNIX systems. Some commands will
+   `differ on Windows OS <https://realpython.com/python-virtual-environments-a-primer/>`_
+
+
+Install Lamassu from Source Locally
+===================================
+
+We recommend creating a virtualenv for your application and activate it
+
+Navigate to the ``lamassu`` root directory and run::
+
+    pip3 install -e .
+
+For more general information, please refer to the
+`Hitchhiker's Guide to Python <https://docs.python-guide.org/writing/structure/#structuring-your-project>`_: "Structuring Your Project".
+
+
+"module 'collections' has no attribute 'Callable' Error When Running nosetests
+==============================================================================
+
+First, uninstall nose with the following command::
+
+    pip3 uninstall -y nose
+
+Second, reinstall nose but with ``--nobinaries`` flag::
+
+    pip3 install -U nose --no-binary :all:
+
+Why does this work? At the time of this writing the binary generated by nose was likely generated with a version of
+Python 3.4 or older. This command forces to rebuild from source.
+
+
+No module named 'pytest' while Running Test Directly in PyCharm
+===============================================================
+
+"Right-click" run a ``test_**.py`` file results in::
+
+    Traceback (most recent call last):
+      File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py", line 5, in <module>
+        import pytest
+    ModuleNotFoundError: No module named 'pytest'
+
+The solution is going to '**Settings** -> **Tools** -> **Python Integrated Tools**' and scroll down to where it says
+`pytest not found` and there is a **FIX** button. Clicking on it and apply the settings shall resolve the problem

docs/source/index.rst

@@ -0,0 +1,46 @@
+=====================
+Lamassu documentation
+=====================
+
+
+Getting help
+============
+
+Having trouble? We'd like to help!
+
+* Try the :doc:`FAQ <faq>` -- it's got answers to some common questions.
+* Looking for specific information? Try the :ref:`genindex` or :ref:`modindex`.
+* Report bugs with lamassu in our `issue tracker`_.
+* Join the Discord community `Lamassu Discord`_.
+
+.. hint::
+    * Since methods with two underscores (`__`) cannot be tested due to the
+      `name mangling <https://qubitpi.github.io/cpython/tutorial/classes.html#private-variables>`_. Lamassu requires
+      all private methods and attributes to be prefixed with **single underscore prefix (`_`) only**
+    * The phrase "Chinese" used throughout this documentation referse to "**Simplified Chinese**", instead of
+      "Traditional Chinese"
+
+
+First Steps
+===========
+
+.. toctree::
+   :caption: First steps
+   :hidden:
+
+   intro/install
+
+:doc:`intro/install`
+    Get lamassu installed on your computer.
+
+
+Usage
+=====
+
+.. toctree::
+   :maxdepth: 100
+
+   lamassu
+
+.. _issue tracker: https://github.com/QubitPi/lamassu/issues
+.. _Lamassu Discord: https://discord.com/widget?id=1208960229002317934&theme=dark

docs/source/intro/install.rst

@@ -0,0 +1,54 @@
+.. _intro-install:
+
+==================
+Installation guide
+==================
+
+
+Supported Python versions
+=========================
+
+Lamassu has been tested with Python 3.10. It may work with older versions of Python but it is not guaranteed.
+
+
+Installing Lamassu
+==================
+
+If you are already familiar with installation of Python packages, we can install Lamassu and its dependencies from
+`PyPI <https://pypi.org/project/lamassu/>`_ with::
+
+    pip3 install lamassu
+
+We strongly recommend that you install Lamassu in :ref:`a dedicated virtualenv <intro-using-virtualenv>`, to avoid
+conflicting with your system packages.
+
+If you're using `Anaconda <https://docs.anaconda.com/anaconda/>`_ or
+`Miniconda <https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html>`_, please allow me to
+apologize because I hate those two, so we won't install the package from there.
+
+
+Installing from Source
+======================
+
+When we want to apply a bug fix quicly by installing Lamassu locally, we can use::
+
+    git clone https://github.com/QubitPi/lamassu.git
+    cd lamassu
+    pip3 install -e .
+
+
+.. _intro-using-virtualenv:
+
+Using a virtual environment (recommended)
+-----------------------------------------
+
+We recommend installing lamassu a virtual environment on all platforms.
+
+Python packages can be installed either globally (a.k.a system wide), or in user-space. We do not recommend installing
+lamassu system wide. Instead, we recommend installing lamassu within a "virtual environment" (:mod:`venv`),
+which keep you from conflicting with already-installed Python system packages.
+
+See :ref:`tut-venv` on how to create your virtual environment.
+
+Once you have created a virtual environment, we can install lamassu inside it with ``pip3``, just like any other
+Python package.

docs/source/lamassu.rst

@@ -0,0 +1,12 @@
+=======
+Lamassu
+=======
+
+
+Machine Learning
+================
+
+.. toctree::
+   :maxdepth: 100
+
+   machine_learning/rnn

docs/source/machine_learning/rnn.rst

@@ -0,0 +1,182 @@
+================================================
+Introduction to Recurrent Neural Networks (RNNs)
+================================================
+
+.. contents:: Table of Contents
+    :depth: 2
+
+
+.. automodule:: lamassu.rnn.vanilla
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+
+Mathematical Formulation
+------------------------
+
+Recurrent neural networks, also known as RNNs, are a class of neural networks that allow previous outputs to be used as
+inputs while having hidden states. They are typically as follows:
+
+.. figure:: ../img/architecture-rnn-ltr.png
+    :align: center
+
+For each timestep :math:`t` the activation :math:`a^{\langle t \rangle}` and the output :math:`y^{\langle t \rangle}` are expressed as follows:
+
+.. math::
+
+    h^{\langle t \rangle} = g_1\left( W_{hh}h^{\langle t - 1 \rangle} + W_{hx}x^{\langle t \rangle} + b_h \right)
+
+    y^{\langle t \rangle} = g_2\left( W_{yh}h^{\langle t \rangle} + b_y \right)
+
+where :math:`W_{hx}`, :math:`W_{hh}`, :math:`W_{yh}`, :math:`b_h`, :math:`b_y` are coefficients that are shared temporally and :math:`g_1`, :math:`g_2` are activation functions.
+
+.. figure:: ../img/description-block-rnn-ltr.png
+    :align: center
+
+A Python implementation of network above, as an example, could be as follows:
+
+.. code-block:: python
+
+    import numpy as np
+    from math import exp
+
+    np.random.seed(0)
+    class VanillaRecurrentNetwork(object):
+
+        def __init__(self):
+            self.hidden_state = np.zeros((3, 3))
+            self.W_hh = np.random.randn(3, 3)
+            self.W_xh = np.random.randn(3, 3)
+            self.W_hy = np.random.randn(3, 3)
+            self.Bh = np.random.randn(3,)
+            self.By = np.random.rand(3,)
+
+            self.hidden_state_activation_function = lambda x : np.tanh(x)
+            self.y_activation_function = lambda x : x
+
+        def forward_prop(self, x):
+            self.hidden_state = self.hidden_state_activation_function(
+                np.dot(self.hidden_state, self.W_hh) + np.dot(x, self.W_xh) + self.Bh
+            )
+
+            return self.y_activation_function(self.W_hy.dot(self.hidden_state) + self.By)
+
+Notice the weight matrix above are randomly initialized. This makes it a "silly" network that doesn't help us anything
+good:
+
+.. code-block:: python
+
+    input_vector = np.ones((3, 3))
+    silly_network = RecurrentNetwork()
+
+    # Notice that same input, but leads to different ouptut at every single time step.
+    print silly_network.forward_prop(input_vector)
+    print silly_network.forward_prop(input_vector)
+    print silly_network.forward_prop(input_vector)
+
+    # this gives us
+    [[-1.73665315 -2.40366542 -2.72344361]
+     [ 1.61591482  1.45557046  1.13262256]
+     [ 1.68977504  1.54059305  1.21757531]]
+    [[-2.15023381 -2.41205828 -2.71701457]
+     [ 1.71962883  1.45767515  1.13101034]
+     [ 1.80488553  1.542929    1.21578594]]
+    [[-2.15024751 -2.41207375 -2.720968  ]
+     [ 1.71963227  1.45767903  1.13200175]
+     [ 1.80488935  1.54293331  1.21688628]]
+
+This is because we haven't train our RNN network yet, which we discuss next
+
+Training
+--------
+
+.. admonition:: Prerequisite
+
+    We would assume some basic Artificial Neural Network concepts, which are drawn from *Chapter 4 - Artificial Neural
+    Networks* (p. 81) of `MACHINE LEARNING by Mitchell, Thom M. (1997)`_ Paperback. Please, if possible, read the
+    chapter beforehand and refer to it if something looks confusing in the discussion of this section
+
+In the case of a recurrent neural network, we are essentially backpropagation through time, which means that we are
+forwarding through entire sequence to compute losses, then backwarding through entire sequence to compute gradients.
+Formally, the `loss function`_ :math:`\mathcal{L}` of all time steps is defined as the sum of
+the loss at every time step:
+
+.. math::
+
+    \mathcal{L}\left( \hat{y}, y \right) = \sum_{t = 1}^{T_y}\mathcal{L}\left( \hat{y}^{<t>}, y^{<t>} \right)
+
+However, this becomes problematic when we want to train a sequence that is very long. For example, if we were to train a
+a paragraph of words, we have to iterate through many layers before we can compute one simple gradient step. In
+practice, for the back propagation, we examine how the output at the very *last* timestep affects the weights at the
+very first time step. Then we can compute the gradient of loss function, the details of which can be found in the
+`Vanilla RNN Gradient Flow & Vanishing Gradient Problem`_
+
+.. admonition:: Gradient Clipping
+
+    Gradient clipping is a technique used to cope with the `exploding gradient`_ problem sometimes encountered when
+    performing backpropagation. By capping the maximum value for the gradient, this phenomenon is controlled in
+    practice.
+
+    .. figure:: ../img/gradient-clipping.png
+        :align: center
+
+    In order to remedy the vanishing gradient problem, specific gates are used in some types of RNNs and usually have a
+    well-defined purpose. They are usually noted :math:`\Gamma` and are defined as
+
+    .. math::
+
+        \Gamma = \sigma(Wx^{<t>} + Ua^{<t - 1>} + b)
+
+    where :math:`W`, :math:`U`, and :math:`b` are coefficients specific to the gate and :math:`\sigma` is the sigmoid
+    function
+
+LSTM Formulation
+^^^^^^^^^^^^^^^^
+
+Now we know that Vanilla RNN has Vanishing/exploding gradient problem, `LSTM Formulation`_ discusses the theory of LSTM
+which is used to remedy this problem.
+
+Applications of RNNs
+--------------------
+
+RNN models are mostly used in the fields of natural language processing and speech recognition. The different
+applications are summed up in the table below:
+
+.. list-table:: Applications of RNNs
+   :widths: 20 60 20
+   :align: center
+   :header-rows: 1
+
+   * - Type of RNN
+     - Illustration
+     - Example
+   * - | One-to-one
+       | :math:`T_x = T_y = 1`
+     - .. figure:: ../img/rnn-one-to-one-ltr.png
+     - Traditional neural network
+   * - | One-to-many
+       | :math:`T_x = 1`, :math:`T_y > 1`
+     - .. figure:: ../img/rnn-one-to-many-ltr.png
+     - Music generation
+   * - | Many-to-one
+       | :math:`T_x > 1`, :math:`T_y = 1`
+     - .. figure:: ../img/rnn-many-to-one-ltr.png
+     - Sentiment classification
+   * - | Many-to-many
+       | :math:`T_x = T_y`
+     - .. figure:: ../img/rnn-many-to-many-same-ltr.png
+     - Named entity recognition
+   * - | Many-to-many
+       | :math:`T_x \ne T_y`
+     - .. figure:: ../img/rnn-many-to-many-different-ltr.png
+     - Machine translation
+
+.. _`exploding gradient`: https://qubitpi.github.io/stanford-cs231n.github.io/rnn/#vanilla-rnn-gradient-flow--vanishing-gradient-problem
+
+.. _`MACHINE LEARNING by Mitchell, Thom M. (1997)`: https://a.co/d/bjmsEOg
+
+.. _`loss function`: https://qubitpi.github.io/stanford-cs231n.github.io/neural-networks-2/#losses
+.. _`LSTM Formulation`: https://qubitpi.github.io/stanford-cs231n.github.io/rnn/#lstm-formulation
+
+.. _`Vanilla RNN Gradient Flow & Vanishing Gradient Problem`: https://qubitpi.github.io/stanford-cs231n.github.io/rnn/#vanilla-rnn-gradient-flow--vanishing-gradient-problem

docs/source/requirements.txt

@@ -0,0 +1,7 @@
+sphinx==5.0.2
+sphinx-hoverxref==1.1.1
+sphinx-notfound-page==0.8
+sphinx-rtd-theme==1.0.0
+pycodestyle
+requests
+pyan3

requirements.txt

@@ -0,0 +1 @@
+numpy

rnn/vanilla.py

@@ -0,0 +1,25 @@
+import numpy as np
+from math import exp
+
+np.random.seed(0)
+class VanillaRecurrentNetwork(object):
+
+    def __init__(self):
+        self.hidden_state = np.zeros((3, 3))
+        self.W_hh = np.random.randn(3, 3)
+        self.W_xh = np.random.randn(3, 3)
+        self.W_hy = np.random.randn(3, 3)
+        self.Bh = np.random.randn(3,)
+        self.By = np.random.rand(3,)
+
+        self.hidden_state_activation_function = lambda x : np.tanh(x)
+        self.y_activation_function = lambda x : x
+
+        self.loss_funciton = lambda y : exp(y) / np.sum(exp(y))
+
+    def forward_prop(self, x):
+        self.hidden_state = self.hidden_state_activation_function(
+            np.dot(self.hidden_state, self.W_hh) + np.dot(x, self.W_xh) + self.Bh
+        )
+
+        return self.y_activation_function(self.W_hy.dot(self.hidden_state) + self.By)

setup.py

@@ -0,0 +1,18 @@
+from setuptools import setup, find_packages
+
+setup(
+    name="lamassu",
+    version="0.0.8",
+    description="Empowering individual to agnostically run machine learning algorithms to produce ad-hoc AI features",
+    url="https://github.com/QubitPi/lamassu",
+    author="Jiaqi liu",
+    author_email="[email protected]",
+    license="Apache-2.0",
+    packages=find_packages(),
+    python_requires='>=3.10',
+    install_requires=[
+
+    ],
+    zip_safe=False,
+    include_package_data=True
+)