Sign in to track progress, unlock prep layers, and browse your pipeline.Sign in
Problem: Stream Processor for Log Tagging
Design a streaming component that processes a single, mixed input stream of messages. Each message is either a query registration or a log line.
Message Types and Processing:
- Query Registration (prefix "Q:"): Defines a string pattern.
- Query IDs are assigned incrementally, starting from 1, in the order they arrive.
- The system must output an acknowledgment (e.g., "ACK <id>") upon registration.
- Log Line (prefix "L:"): Contains text that needs to be tagged.
- For each log line, the system must identify and tag it with the IDs of all currently active queries whose pattern appears as a case-sensitive substring within the log text.
- The system must output the original log line followed by the list of matching query IDs, sorted in ascending order (e.g., "L: <log_text> | matches: [<id1>, <id2>]").
Example:
Consider the following input stream and expected outputs:
Input Stream:
- Q: error
- Q: timeout
- L: database timeout after 5s
- L: ERROR: connection reset
- Q: reset
- L: timeout reset error
Expected Outputs:
- For 1) → ACK 1
- For 2) → ACK 2
- For 3) → L: database timeout after 5s | matches: [2]
- For 4) → L: ERROR: connection reset | matches: [3] (Note: "ERROR" does not match "error" due to case-sensitivity)
- For 5) → ACK 3
- For 6) → L: timeout reset error | matches: [1, 2, 3]
Tasks:
- Initial Design & Implementation: Design and describe a function or process that handles query ID assignment and log tagging as specified.
- Scalability: Propose modifications to your design to efficiently handle a very large volume of data, encompassing both queries and log lines.
- Query Deletion: Describe how to support the deletion of registered queries in your current implementation, and identify any inefficiencies introduced by this feature that would need to be addressed.
Assumptions:
- Matching is based on case-sensitive substring search. The design should note how to extend this to case-insensitive matching or regular expressions.
- Queries only apply to log lines that arrive after their registration; no retroactive tagging is required.
- In-order processing of the single input stream is sufficient for correctness. However, candidates should be prepared to discuss scaling beyond a single process.
AI interview prep
Core dilemma
Efficiently matching a high-volume stream of logs against a large, dynamically changing set of string patterns while maintaining low latency and high throughput.
Mid-level signals3
Senior signals3
Red flags3
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Problem: Stream Processor for Log Tagging
Design a streaming component that processes a single, mixed input stream of messages. Each message is either a query registration or a log line.
Message Types and Processing:
- Query Registration (prefix "Q:"): Defines a string pattern.
- Query IDs are assigned incrementally, starting from 1, in the order they arrive.
- The system must output an acknowledgment (e.g., "ACK <id>") upon registration.
- Log Line (prefix "L:"): Contains text that needs to be tagged.
- For each log line, the system must identify and tag it with the IDs of all currently active queries whose pattern appears as a case-sensitive substring within the log text.
- The system must output the original log line followed by the list of matching query IDs, sorted in ascending order (e.g., "L: <log_text> | matches: [<id1>, <id2>]").
Example:
Consider the following input stream and expected outputs:
Input Stream:
- Q: error
- Q: timeout
- L: database timeout after 5s
- L: ERROR: connection reset
- Q: reset
- L: timeout reset error
Expected Outputs:
- For 1) → ACK 1
- For 2) → ACK 2
- For 3) → L: database timeout after 5s | matches: [2]
- For 4) → L: ERROR: connection reset | matches: [3] (Note: "ERROR" does not match "error" due to case-sensitivity)
- For 5) → ACK 3
- For 6) → L: timeout reset error | matches: [1, 2, 3]
Tasks:
- Initial Design & Implementation: Design and describe a function or process that handles query ID assignment and log tagging as specified.
- Scalability: Propose modifications to your design to efficiently handle a very large volume of data, encompassing both queries and log lines.
- Query Deletion: Describe how to support the deletion of registered queries in your current implementation, and identify any inefficiencies introduced by this feature that would need to be addressed.
Assumptions:
- Matching is based on case-sensitive substring search. The design should note how to extend this to case-insensitive matching or regular expressions.
- Queries only apply to log lines that arrive after their registration; no retroactive tagging is required.
- In-order processing of the single input stream is sufficient for correctness. However, candidates should be prepared to discuss scaling beyond a single process.