A financial institution is developing an Azure AI solution that uses Azure Cognitive Search to index and search millions of financial documents. The documents contain highly sensitive Personally Identifiable Information (PII) and compliance regulations mandate that this PII must be masked or redacted before being stored in the search index, but the original documents must remain accessible in their raw form in a secure data lake. Which Cognitive Search capability, combined with Azure Functions, should be leveraged during the indexing pipeline to achieve this requirement?
- AIndexer with a built-in text analytics skill for language detection.
- BSkillset with an Entity Recognition skill.
- CSkillset with a Custom Skill that calls an Azure Function for PII masking.
- DData source configuration with field mapping for direct redaction.
Show answer & explanationAnswer & explanation
Correct answer: C. Skillset with a Custom Skill that calls an Azure Function for PII masking.
A Cognitive Search skillset with a Custom Skill is the most appropriate solution. A Custom Skill can invoke an Azure Function, which can then implement custom logic to detect and mask/redact PII from the document content before it's indexed, while the original document remains untouched in the data source. This allows for tailored redaction rules and integration with other PII detection libraries.
Why the other options are wrong
- A. A built-in text analytics skill for language detection is useful for processing, but it does not perform PII masking or redaction.
- B. An Entity Recognition skill can detect PII, but it doesn't inherently mask or redact it from the indexed content; it merely identifies its presence.
- D. Field mapping primarily controls how source fields are mapped to index fields and basic transformations, but it cannot perform complex, conditional PII masking or redaction logic dynamically.
Cognitive Search Custom Skill
A Cognitive Search Custom Skill extends the AI enrichment pipeline by integrating external logic, often via Azure Functions, to perform specialized processing like PII masking or custom entity extraction.
- Invokes external web APIs or Azure Functions.
- Enables custom data transformation and enrichment.
- Integrates seamlessly into the Cognitive Search indexing pipeline.
Memory trick: Custom Skills 'Mask' PII with Function's help.