RAGchain.benchmark.extra package

Submodules

RAGchain.benchmark.extra.search_detector_evaluator module

class RAGchain.benchmark.extra.search_detector_evaluator.SearchDetectorEvaluator(search_detector: RedeSearchDetector, random_state: int = 42)

Bases: object

You can evaluate search detector performance using this class. This class uses DSTC11 Track5 dataset for evaluation. And used metrics for evaluation are precision, recall, f1. From now, we only support RedeSearchDetector, because it is only Search Detector that supports in RAGchain. You might use more search detector when we add them.

evaluate(test_cnt: int = 500)
extract_last_log_text(row)
train(train_knowledge_seeking_cnt: int = 5, train_non_knowledge_seeking_cnt: int = 50, valid_percentage: float = 0.2)

Module contents