Financial Services and Insurance

[We used Cleanlab in] an update of one of the functionalities offered by the BBVA app: the categorization of financial transactions. These categories allow users to group their transactions to better control their income and expenses, and to understand the overall evolution of their finances. This service is available to all users in Spain, Mexico, Peru, Colombia, and Argentina.

We used AL [Active Learning] in combination with Cleanlab.

This was necessary because, although we had defined and unified annotation criteria for transactions, some could be linked to several subcategories depending on the annotator’s interpretation. To reduce the impact of having different subcategories for similar transactions, we used Cleanlab for discrepancy detection.

With the current model, we were able to improve accuracy by 28%, while reducing the number of labeled transactions required to train the model by more than 98%.

CleanLab assimilates input from annotators and corrects any discrepancies between similar samples.

CleanLab helped us reduce the uncertainty of noise in the tags. This process enabled us to train the model, update the training set, and optimize its performance. The goal was to reduce the number of labeled transactions and make the model more efficient, requiring less time and dedication. This allows data scientists to focus on tasks that generate greater value for customers and organizations.

We used cleanlab for finding label errors in financial text data, helping us find label errors in our human annotation process. I like cleanlab more than alternative solutions because it's like 'bring your own model' & 'bring your own data', by acting like a wrapper around your model, it's superbly easy to implement, and it works well even when the model itself is not decent in classification due to relatively high noise rate (40% noisy) to achieve a consistent f1 score around 80%.