MarkItDown is a utility for converting various files to Markdown, including PDF, PowerPoint, Word, Excel, Images, Audio, HTML, text-based formats, and ZIP files.
An open-source project offering a functional RAG UI for document QA, suitable for both end-users and developers. It supports various LLM providers, is customizable, and offers multi-modal QA, citations, and complex reasoning methods.
pip install 'ragna builtin » ' # Install ragna with all extensions ragna config # Initialize configuration ragna ui # Launch the web app
A simple Python3 tool to detect similarities between files within a repository. Document similarity code adapted from Jonathan Mugan's tutorial: https://www.oreilly.com/learning/how-do-i-compare-document-similarity-using-python '''
tokenizing and stemming each synopsis transforming the corpus into vector space using tf-idf calculating cosine distance between each document as a measure of similarity clustering the documents using the k-means algorithm using multidimensional scaling to reduce dimensionality within the corpus plotting the clustering output using matplotlib and mpld3 conducting a hierarchical clustering on the corpus using Ward clustering plotting a Ward dendrogram topic modeling using Latent Dirichlet Allocation (LDA)
