FastEHR.dataloader.tokenizers_local.base
========================================

.. py:module:: FastEHR.dataloader.tokenizers_local.base


Classes
-------

.. autoapisummary::

   FastEHR.dataloader.tokenizers_local.base.TokenizerBase


Module Contents
---------------

.. py:class:: TokenizerBase

   Base class for custom tokenizers


   .. py:property:: vocab_size


   .. py:property:: fit_description


   .. py:method:: event_frequency(meta_information, include_measurements=True, include_diagnoses=True) -> polars.DataFrame
      :staticmethod:


      Get polars dataframe with three columns: event, count and relative frequencies

      Returns
      ┌──────────────────────────┬─────────┬───────────┐
      │ EVENT                    ┆ COUNT   ┆ FREQUENCY │
      │ ---                      ┆ ---     ┆ ---       │
      │ str                      ┆ u32     ┆ f64       │
      ╞══════════════════════════╪═════════╪═══════════╡
      │ <event name 1>           ┆ n1      ┆ p1        │
      │ <event name 2>           ┆ n2      ┆ p2        │
      │ …                        ┆ …       ┆ …         │
      └──────────────────────────┴─────────┴───────────┘



   .. py:method:: fit(event_counts: polars.DataFrame, **kwargs)
      :abstractmethod:


              



   .. py:method:: encode(sequence: list[str])

      Take a <> of strings, output a list of integers



   .. py:method:: decode(sequence: list[str])