What makes an AI project production ready?
In traditional software development the checklist for a project before it is put in production might include:
- security reviews - double check adherence to security best practices and relevant policies
- monitoring and logging - ensure performance issues and errors in the front-end and back-end are tracked so they can be addressed
- scalability - identify plans for meeting and exceeding expected load
- rollout and upgrade plans - plan for how maintainance, upgrades, and rollbacks will be performed on the system
- disaster recovery - ensure data and database backups are available and can actually be used to recover a system
How about an AI based system? What additional items should be on the list? Perhaps:
- final model accuracy check - does the latest approved model version meet the quality expectations
- data quality check - are we sure of the data quality the model was trained on
- data drift monitoring - when the world changes does the model continue to perform as desired
- bias and ethical considerations - have the appropriate parties examined the model, data and outputs and approved it for this use
- explainability and transparency compliance - have interested parties signed off on the model meeting legal and business requirements.
- reproducibility - do we have enough meta-data on the entire system (model and data ids, architecture description, etc) to rebuild the system if (when) a problem is detected.
What else can you think of?
Want to get notified of new articles and insights?