A Long-term VIS-enabled Infrastructure for Supporting ML-assisted Human Decision-making

Many large organisations maintains a large pool of trained human resources. When a new task arrives, the management constructs a team by selecting appropriate team members with different skills and arranges an effective operational structure for the team. In machine learning (ML), the model developers typically train many models for each individual task, then select the best model to perform the task, while discarding the unselected models. Considering that keeping a trained ML model costs much less than employing a person, there is a huge waste of model resources. The main reasons behind this wasteful practice include (i) the lack of effective means for apprehending the "skill profiles" a large number of ML models; (ii) the lack of effective means for constructing a "team" such that the combined skillset of the team is suitable for the task but each component model does not have all the skills required; and (iii) the lack of effective means for enabling human decision makers to utilise imperfect ML models as assistants or advisers. Because of these reasons, there is less incentive to maintain a large pool of trained ML models that may not be the best for a specific task individually, and the emphasis has been placed on training a "star" model as optimal as possible for each arrival task.

The technology of visualization and visual analytics (VIS) can address the aforementioned three "lacks". In many data-intensive applications, VIS can enable decision-makers to observe a large amount of data quickly (e.g., stock market), analyse complex relationships among different data entities (e.g., social network analysis), and make complex judgement based on multiple and sometimes conflicting machine-predictions (e.g., by different epidemiological models). The latest theoretical advance offers an explanation as to what visualization offers users that statistics and algorithms cannot offer. Humans have limited cognitive bandwidth for...