RAGchain.reranker.tart package
Submodules
RAGchain.reranker.tart.modeling_enc_t5 module
- class RAGchain.reranker.tart.modeling_enc_t5.EncT5ForSequenceClassification(config: T5Config, dropout=0.1)
Bases:
T5PreTrainedModel- deparallelize()
- forward(input_ids=None, attention_mask=None, head_mask=None, inputs_embeds=None, labels=None, output_attentions=None, output_hidden_states=None, return_dict=None)
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Moduleinstance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- get_encoder()
- get_input_embeddings()
Returns the model’s input embeddings.
- Returns:
nn.Module: A torch module mapping vocabulary to hidden states.
- parallelize(device_map=None)
- set_input_embeddings(new_embeddings)
Set model’s input embeddings.
- Args:
value (nn.Module): A module mapping vocabulary to hidden states.
- training: bool
RAGchain.reranker.tart.tart module
- class RAGchain.reranker.tart.tart.TARTReranker(instruction: str)
Bases:
BaseRerankerTARTReranker is a reranker based on TART (https://github.com/facebookresearch/tart). You can rerank the passages with the instruction using TARTReranker.
- invoke(input: Input, config: RunnableConfig | None = None) Output
Transform a single input into an output. Override to implement.
- Args:
input: The input to the runnable. config: A config to use when invoking the runnable.
The config supports standard keys like ‘tags’, ‘metadata’ for tracing purposes, ‘max_concurrency’ for controlling how much work to do in parallel, and other keys. Please refer to the RunnableConfig for more details.
- Returns:
The output of the runnable.
RAGchain.reranker.tart.tokenization_enc_t5 module
- class RAGchain.reranker.tart.tokenization_enc_t5.EncT5Tokenizer(vocab_file, bos_token='<s>', eos_token='</s>', unk_token='<unk>', pad_token='<pad>', extra_ids=100, additional_special_tokens=None, sp_model_kwargs: Dict[str, Any] | None = None, **kwargs)
Bases:
T5Tokenizer- build_inputs_with_special_tokens(token_ids_0: List[int], token_ids_1: List[int] | None = None) List[int]
Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and adding special tokens. A sequence has the following format: - single sequence: <s> X </s> - pair of sequences: <s> A </s> B </s> Args:
- token_ids_0 (List[int]):
List of IDs to which the special tokens will be added.
- token_ids_1 (List[int], optional):
Optional second list of IDs for sequence pairs.
- Returns:
List[int]: List of [input IDs](../glossary#input-ids) with the appropriate special tokens.
- create_token_type_ids_from_sequences(token_ids_0: List[int], token_ids_1: List[int] | None = None) List[int]
Create a mask from the two sequences passed to be used in a sequence-pair classification task. T5 does not make use of token type ids, therefore a list of zeros is returned. Args:
- token_ids_0 (List[int]):
List of IDs.
- token_ids_1 (List[int], optional):
Optional second list of IDs for sequence pairs.
- Returns:
List[int]: List of zeros.
- get_special_tokens_mask(token_ids_0: List[int], token_ids_1: List[int] | None = None, already_has_special_tokens: bool = False) List[int]
Retrieve sequence ids from a token list that has no special tokens added. This method is called when adding special tokens using the tokenizer prepare_for_model method. Args:
- token_ids_0 (List[int]):
List of IDs.
- token_ids_1 (List[int], optional):
Optional second list of IDs for sequence pairs.
- already_has_special_tokens (bool, optional, defaults to False):
Whether or not the token list is already formatted with special tokens for the model.
- Returns:
List[int]: A list of integers in the range [0, 1]: 1 for a special token, 0 for a sequence token.