What makes an AI project production ready?

In traditional software development the checklist for a project before it is put in production might include:

  • security reviews - double check adherence to security best practices and relevant policies
  • monitoring and logging - ensure performance issues and errors in the front-end and back-end are tracked so they can be addressed
  • scalability - identify plans for meeting and exceeding expected load
  • rollout and upgrade plans - plan for how maintainance, upgrades, and rollbacks will be performed on the system
  • disaster recovery - ensure data and database backups are available and can actually be used to recover a system

How about an AI based system? What additional items should be on the list? Perhaps:

  • final model accuracy check - does the latest approved model version meet the quality expectations
  • data quality check - are we sure of the data quality the model was trained on
  • data drift monitoring - when the world changes does the model continue to perform as desired
  • bias and ethical considerations - have the appropriate parties examined the model, data and outputs and approved it for this use
  • explainability and transparency compliance - have interested parties signed off on the model meeting legal and business requirements.
  • reproducibility - do we have enough meta-data on the entire system (model and data ids, architecture description, etc) to rebuild the system if (when) a problem is detected.

What else can you think of?

Want to get notified of new articles and insights?