Code

Anchors

gym-gridverse - Custom Gridworld Environments for Reinforcement Learning

Gridworld domains for fully and partially observable reinforcement learning.

GridVerse is highly customizable; while many components are provided out-of-the-box, it is designed such that you can create your own components programmatically, including your own objects, starting states, transition functions, reward functions, observation functions, terminating functions, etc.

Available at abaisero/gym-gridverse.

asym-rlpo - Asymmetric Reinforcement Learning under Partial Observability

Research code repository.

Relevant Publications
paper slides poster video Baisero, Daley, and Amato, “Asymmetric DQN for Partially Observable Reinforcement Learning,” in Proceedings of the Conference on Uncertainty in Artificial Intelligence, 2022.
paper slides poster video Baisero and Amato, “Unbiased Asymmetric Reinforcement Learning under Partial Observability,” in Proceedings of the Conference on Autonomous Agents and Multiagent Systems, 2022.
Available at abaisero/asym-rlpo.

rl-psr - Reward-Predictive State Representations

Research code repository.

Relevant Publications
paper slides poster video Baisero and Amato, “Reconciling Rewards with Predictive State Representations,” in Proceedings of the International Joint Conference on Artificial Intelligence, 2021.
Available at abaisero/rl-rpsr.

Learning Internal State Models in Partially Observable Environments

Research code repository.

Relevant Publications
paper slides poster video Baisero and Amato, “Learning Internal State Models in Partially Observable Environments,” in Reinforcement Learning under Partial Observability, NeurIPS Workshop, 2018.
Available at abaisero/2018-nips-rlpo-code.

gym-pomdps - Gym Environments for Flat POMDPs

Gym environments for POMDPs and DecPOMDPs respectively encoded by .POMDP and .DPOMDP files.

Available at abaisero/gym-pomdps.

rl-parsers - Reinforcement Learning Parsers

Parsers for file formats related to reinforcement learning, including the standard .MDP, .POMDP and .DPOMDP file formats, and the custom .FSC (Finite State Controller) and .FSS (Finite State Structure) file formats.

Available at abaisero/rl-parsers.

one-to-one - Tools for Semantic Indexing

Tools for flat indexing of complicated semantic structures.

Family of classes which represent bijections between sets of \(N\) structured and/or semantic values/objects and a corresponding set of indices \({0, \ldots, N-1}\); Useful to represent and access tables of values indexed directly by objects with rich interfaces and attributes.

The provided classes automate (via a user-friendly interface) the conversions between values and indices.

Available at abaisero/one-to-one.

Website

This website is built with Jekyll, a minimalistic static website generator that uses HTML, Markdown, and Liquid to define page templates and fill in content. The website style is powered by Bootstrap (an HTML/CSS/JS framework) and Font Awesome (an icon toolkit). Despite not being a web developer, these tools allowed me to create this website with ease.

Available at abaisero/website.