The internet has grown to unimaginable levels that there is a large influx of data in various formats like documents and images from all sorts of sources. Given that these documents are easily accessible via mobile devices, we need to ensure that the growing volume of stored documents are managed well. To focus on the speed and efficiency of document classification in resource limited environments, this paper presents a novel On-Device Document Classification System capable of real-time processing, combining Optical Character Recognition (OCR), Convolutional Neural Networks (CNNs), and FuzzyWuzzy logic. Given the limitations of existing datasets, we curated a custom-labeled dataset for training EfficientNet and MobileNet models. We essentially combine visual and textual features through a OCR with FuzzyWuzzy logic, a CNN, and a voting algorithm to make a final classification. For this, our pipeline standardizes the input by converting documents to images, extracting text with EasyOCR, and using TF-IDF vectors as the input for our CNN. To take error into account, we utilize FuzzyWuzzy logic to enhance accuracy by matching text with class keywords. This method ensures that we receive fast and accurate classification on edge devices, preserving data privacy. It achieves 97.1% accuracy on RVL-CDIP and MASK-RCNN datasets, with real-time processing in 2.3 seconds.