For NLP Projects, how do you keep your end-users engaged from day 1?
The most challenging job in any enterprise NLP project is to convince field experts to train the bot. Since a good outcome will replace many of them. So, for the ML (machine learning) team, how do you overcome the initial hurdle – getting these field experts to train your bot?
You can (we don’t recommend) –
- Use the top-down approach, use management to pressure people to do the work. The downsides are quality and troop morale.
- Hire external firms, consultants, to train the engine or buy off-the-shelf AI models. The downside is that the results may not give you a competitive edge.
Or, You Can (we recommend) –
- You can create an incentive program to make training more bearable and more engaging from day 1.
Sure, the third option sounds promising, but how do you do it?
We recommend the following:
- Provide value from day one via offering better search experience
- Encourage criticism, feedback, and innovation
- Recognize employees who go above and beyond
- Commit to the future success of your employees
Provide value from day 1
Pain Point
A customized NLP project will likely take a year before delivering any tangible output. So, there is a high sunk cost associated with these projects. Since the outcome is not probable, the risk-reward parity is skewed.
Off-the-shelf Approach
We offer hosted elasticsearch environments with datasets covering SEC filings such as 10-K, 10-Q, US public companies’ quarterly transcripts, and major industry conferences. The benefit of choosing us is that you have a deliverable from day 1 – a super fast searchable database that delivers insights to your end-users while training their customizable models. A similar hosted environment could cost up to 3K per seat.
Internal Developed Approach
We recommend building three separate data nodes with replicas, nightly refresh, and backups.
Node 1: Documents with word-level indexation
Node 2: Sectionized documents (in case the data covers a variety of topics and you wish to split up the papers into smaller units)
Node 3: Document split into sentences and then deduped. Unitize exercise is necessary to perform category labeling.
Encourage criticism, feedback, and innovation in your NLP training process
Pain Point
The ML team’s assumption where the business team will provide neatly formatted training and the testing dataset is a pie in the sky.
Off-the-shelf Approach
Instead, wait for field experts to provide you with the training dataset. We recommend using our “incremental training” UI to train and capture trainers’ feedback and innovations via our built-in collaboration module. Hence, the entire training process is under continuous improvement.
Internal Developed Approach
We suggest building features that allow everyone to leave a note for someone else. We also recommend allowing field experts the flexibility to add, rename, remove, or even move sub-categories from one category to another. This way, nothing has to be set in stone on day one of the training. A pitfall for many projects is that teams spend too much time on requirement gathering to discover later that it requires significant changes. Instead, we recommend building models that can be reshaped continuously and remolded based on end-users feedback and what they learned from the data.
III. Recognize employees who go above and beyond
Pain Point
Creating and preparing the training dataset can take time and effort. Unfortunately, such a step often goes unnoticed outside of the training team. The misaligned incentive program can create toxic environments between departments and cause high turnovers to amputate the company’s everyday operations.
Off-the-shelf Approach
We recommend using our training progress reporting package. In this package, you will find many different reports that you can use to track both the quality and the quantity of your team’s training efforts.
Internal Developed Approach
We suggest including at least two matrices in the weekly executive dashboards that show the quality and quantity of each training participants’ team and add cross-validation within groups to ensure quality and void quantitative-only driven behaviors. Make these reports transparent among teams and provide a tangible reward for those who performed.
Commit to the future success of your employees
Paint Point
Most employees have a distaste for training their career replacement bot. Despite how you spin the story, it is evident that some of them will lose their job. How do you make it so that they can be more knowledgeable and ready to tackle more challenging roles after training the bot?
If you can make the training the best stepping stone for their next career move, you might have something to motivate them.
Off-the-shelf Approach
We recommend tieing the training exercise with trainers’ specific career objectives. We believe this approach can increase their loyalty to the project.
Internal Developed Approach
We suggest the team set up exemplary case tagging with detailed case descriptions and explanations. This approach can help quickly train new staff members and give them concrete examples to link training back to their improvement agendas, such as preparing for CFA exams.
Either off-the-shelf or internally developed, we believe the most important thing to remember is to keep everyone engaged and motivated via continuous improvement.
