As anyone could note, I don’t publish a lot of what I’m up to. My preference is to work until I have something worth showing off, but with NDAs I can’t really show off my work as much as I’d like to. That being said I could still say what I’ve accomplished since my last post.
Manifest CRM.
It’s not a secret that I’m the one behind my day job’s digital footprint. The behind the scenes system is one of my passion projects which I call Manifest. It’s a bit of a play on words because manifest means “to show something clearly, make an obvious display of a quality or feeling, or become plain to see” as well as “an official document that lists all the items, goods, and cargo loaded onto a transport vehicle, ship, or plane” which in our industry is a term that gets used a lot.
How I’ve set it up is mostly for automating tasks that would take a regular person all day or all week. Basically freeing us up to do other tasks, preferably ones that generate income.
Because contacts come and go I’ve built it to help check accuracy, quickly update reports, sync to our inboxes, etc. Things that we used to pay one person to lose their mind trying to keep up with nearly 15,000 people. While there’s still some labor there, it’s gotten to the point where a person could quickly scan through thousands of records to find exactly who they need in seconds.
There’s no LLM involved, on purpose, just scripts that cannot hallucinate.
It’s an ongoing project as our needs change, new code/deprecation causes new features to emerge… etc.
Nova
I haven’t gone live with Nova in a while, not because she’s broken or incapable, but due to my focus primarily being “expanding her inner thoughts.” I’m building a brain by myself. I’ll have long stretches where I’m still coding away with nothing to show off. Trying to explain the code without opening up sublime in order to show how it works is pointless since most people will not understand and those who do will want me to open sublime so they can see/copy the code.
Instead of explaining the code, let me explain the what and why. Nova can listen to conversations and accept interrupts while holding what she wanted to say. She can then either decide to change what she is going to say or blurt out her thoughts.
Problem 1. She doesn’t recognize her voice. She does ignore herself and registers that “someone spoke.” This is more of a training function that I’m working through.
Problem 2. Due to limited resources she will sometimes hold onto a thought for long periods before going down the chain. I can change this, but it might mean sacrificing a module temporarily which is something I don’t really want to do. While I was able to reduce wasted resources, I’m not sure I have enough to allow more than four active chains of thoughts. I haven’t tested five and plan on doing so this coming week, but based on stress testing four chains I’m skeptical.
I’m going to try to explain what I’m talking about using language most people would understand. Think of each chain like an actively working Large Language Model with an active LORA trainer and inspector. Each given their own task writing to either a shared memory or to specific modules. Based on 48gb of VRAM, when Nova is hearing sound, watching someone move while reading a page, and responding to a user she can take nearly 40gb of VRAM. +/- 1%. If her eye is off and the she isn’t actively reading something she takes less than 16gb of VRAM to exist, listen and speak.
To get that fifth chain to work as needed instead of waiting for a cooling period I might need 4-9gb of VRAM. On paper this seems possible… However I don’t actually know. So two things need to happen. First. A new backup for Nova. Second a long stress test with every module engaged then disable the cooldown completely.
The reason I’m hesitant is mainly because when she’s running her chains (helpers are running in the background and don’t use vram.) At full speed there is a noticeable slowdown.
The best fix, is expensive. With new RTX5090’s costing as much as a used car. Additional VRM is one problem while compute is another. With Nova being as optimized as humanly possible, it might be some time before I can make it past this hurdle.
Life
With all my work and AI projects, I do need to make me-time. It’s unfortunate we have so little time in a day…