Technical Product Manager (AI / Platform) Interview Questions and Answers

I treat developers as customers, because a public API is a promise and breaking it costs them real engineering time. I use semantic versioning so the meaning of a change is clear, and I default to additive changes, such as new optional fields, that do not break existing integrations.

When a breaking change is unavoidable, I document the rationale, publish a migration guide with code examples and give a long deprecation window. For a widely used API that is typically twelve months or more, and longer for enterprise integrations that go through slow release cycles.

Usage telemetry is what makes this responsible. I track which versions, endpoints and customers are still active, so I can contact the people affected directly rather than hoping they read a changelog.

I only remove something once usage is near zero or the remaining customers have a supported path. Breaking changes without notice destroy trust that took years to build.

I write requirements that describe the problem and the success criteria, and I leave the architecture to the ML engineers. A good spec has six elements.

The task, stated plainly: for example "classify support tickets into twelve categories". The input and output contract: what goes in, what comes back, format, confidence scores and how errors are reported. The eval dataset: real, representative examples with agreed labels, ideally built together. Target metrics per segment, not just overall, because an average can hide a failure on an important customer group. Latency and cost limits, such as p95 under 800 ms and cost under a set amount per thousand requests. And the acceptable failure behaviour, such as fall back to a human.

I stay out of model choice. If I prescribe an architecture, I take responsibility for an outcome I cannot control. Vague criteria like "should be accurate" are the real problem, because they let everyone claim success.

I choose by the nature of the knowledge and the behaviour I need to change. Retrieval-augmented generation suits knowledge that changes often or is proprietary, such as product documentation, policies or customer data. It lets the model cite sources, it can be updated by changing documents rather than retraining, and it keeps data access controlled.

Fine-tuning suits stable behaviour changes: a consistent tone, a strict output format, a specialised task pattern or reducing prompt length at scale. It does not reliably teach new facts, and it needs training data, evaluation and a retraining process.

My default is to start with prompting and RAG because they are cheaper and faster to iterate, and they show quickly whether the use case works. I move to fine-tuning when evidence shows prompts cannot reach the quality target or the prompt cost at volume is too high.

RAG stops being enough when the failure is about reasoning or style rather than missing knowledge, or when retrieval quality becomes the ceiling.

I would not assume a mandate will fix it. Mandated adoption produces compliance on paper and workarounds in practice, and it hides the real problem.

I interview the teams, both the ones using the platform and the ones who avoided it, and ask them to walk me through how they would build their last project. The usual causes are poor documentation, missing features they need, unclear support, or the fact that calling a vendor API directly is simply faster to start with.

I treat internal teams as customers and remove friction in order of impact: better quickstarts and samples, faster access and onboarding, the missing feature if many teams need it, and a visible support channel. The metric I watch is time-to-first-call, along with the number of teams active each month and their repeat usage.

If after that the platform is still not chosen, I take it seriously as a signal that the product itself is not good enough.

I derive targets from what consumers actually need rather than copying another service's numbers. A chat feature needs low time-to-first-token. A nightly batch job cares about throughput and cost. So I talk to the main consuming teams and set latency and availability targets, for instance p95 latency and a monthly availability percentage, for each use case.

Rate limits then come from capacity and unit cost: what the infrastructure can safely serve and what we can afford. I use tiered quotas so that shared infrastructure is protected and heavy users are routed to higher-priced or dedicated capacity.

Burst behaviour is explicit in the documentation: what happens above the limit, how clients should back off, and which response codes and headers they will see.

When a customer needs more, I look at the use case first. Sometimes the answer is a higher tier, sometimes batch endpoints, sometimes reserved capacity, and sometimes a conversation about caching on their side.

Silent behaviour changes are one of the worst failure modes on an AI platform, because downstream teams do not know their prompts and parsing logic are suddenly failing.

I version every model and pin consumers to a specific version by default, so nothing changes under them without their consent. Before any new version rolls out, I run regression evals on prompts and tasks that matter to consumers, comparing old and new outputs, and I include format stability checks like JSON validity.

Rollout is staged. Consumers receive advance notice with a summary of expected differences, a canary window where they can test with a portion of traffic, and a clear rollback path to the previous version for a defined period.

I avoid auto-upgrading everyone. It is convenient for the platform team and expensive for everybody else. A regression eval should include representative consumer prompts, edge cases, safety tests and cost and latency comparisons.

Uptime alone tells me the service responds, not whether it is useful. For a production AI service I monitor several layers.

Operational: latency percentiles, error rates, throughput and availability. Cost: cost per request and per tenant, because a quiet cost spike can be as damaging as an outage. Input drift: changes in what users are sending, which can quietly reduce accuracy. Output quality: automated checks plus a regular sample of outputs reviewed by humans. Safety: guardrail trigger rates and policy violations. And user feedback signals such as thumbs-down or edit rates.

Alerts are tied to thresholds we agreed in advance with the consuming teams, and I separate pages that wake someone from signals that go into a weekly review.

The alert I would page for is a sudden jump in harmful-output rate or a sustained error rate breach. Watching only latency means you can serve wrong answers quickly and not notice.

I favour a paved road. The central platform covers what everybody needs and nobody should rebuild: authentication and security controls, cost tracking, logging, evaluation tooling, guardrails and model access. Using it should be the easiest option, not a requirement.

Teams can deviate when they have a justified need, such as a specialised model, a latency constraint or an unusual data requirement. In that case they take on the responsibility for meeting the same security, privacy and cost standards, and they tell us so the platform learns from the exception.

This avoids two failures. If every team builds its own stack, we get duplicated effort, inconsistent security and uncontrolled spend. If the platform blocks all deviation, it becomes a bottleneck and teams route around it, often invisibly.

I allow an exception when the team can show a concrete requirement the platform does not meet, and I treat repeated exceptions as a roadmap signal.

The most important idea is time to first success. A developer evaluating an API forms a view within minutes, so I design the first experience carefully.

That means a clear quickstart that works with copy and paste, sample projects in the major languages, sandbox keys that need no sales conversation, predictable and descriptive error messages, and honest documentation on rate limits, pricing and known limitations. Hiding limits only creates support tickets and mistrust later.

I treat documentation as part of the product, with an owner, a roadmap and testing. Samples are run in CI so they do not rot.

To measure it, I track time-to-first-call, onboarding drop-off at each step, and support tickets grouped by theme. Themes tell me exactly which part of the experience is failing. I also watch how long it takes a new developer to reach production, not just a first call.

I start by finding where the cost comes from. I break inference spend down by model, customer, feature and request type, and compare cost per request against usage growth. Costs rising faster than usage usually point to longer prompts, a shift to more expensive models, retries, or a few heavy customers.

Then I apply levers in order of how little they harm quality. Caching repeated or near-identical requests. Trimming prompts and context. Routing simple requests to smaller, cheaper models and keeping large models for hard cases. Batching non-urgent work. Quota tiers or pricing that reflect actual cost.

I share cost dashboards with the consuming teams, because people optimise what they can see. Teams that see the cost of their own prompts usually find savings themselves.

I would not cut quality first, because that damages the product to fix a problem that cheaper engineering often solves. I validate every cost lever against the eval set.

I would treat security and privacy as launch requirements, agreed with security and legal before any customer touches the platform.

The core list: tenant data isolation so one customer's data can never reach another; encryption in transit and at rest; a clear policy that customer data is not used for model training without explicit consent; strong access controls and role-based permissions; audit logs for who accessed what; PII detection and redaction where appropriate; and retention limits so data is not kept longer than needed.

AI adds specific threats. I plan defences against prompt injection, especially when models read external content or call tools, and against data leakage through outputs. I also consider regional data requirements and vendor sub-processor terms.

Before launch, there is a named sign-off from security, legal and the platform owner, with a documented risk review. Training on customer data by default is the one choice that can end enterprise trust overnight.

I build evaluation as a shared product with a reusable harness, because otherwise each team creates its own tests and nobody can compare results.

The framework has several parts: versioned datasets so results are reproducible; task-specific metrics, since accuracy for classification is different from faithfulness for summarisation; human-review workflows for judgements that cannot be automated; and automated regression runs in CI so every prompt or model change is tested before release.

Teams contribute their own test cases through a simple process, such as a pull request or a form, and those cases are added to a shared library with tags. A dashboard compares models, prompts and configurations side by side on quality, cost and latency.

This lets a team answer "is the new model better for us?" in an afternoon rather than a month. The risk to watch is stale datasets, so I schedule regular refreshes with fresh production samples.

I design routing around the properties of the request: task type, complexity, cost sensitivity and latency need. A short classification request does not need the largest model, while a complex multi-step analysis might.

My default is a cheap, fast model, with escalation to a more capable one when the request is hard, when confidence is low or when a quality check fails. Routing rules are configurable rather than hard-coded, so teams can adjust them without a release, and every route has a fallback so a provider outage does not become our outage.

I validate rules with data. I run the same evaluation set through candidate routes and compare quality, cost and latency, then monitor in production for drift, because model behaviour changes.

The mistake I try to avoid is hard-coding one model choice in application code. It makes cost optimisation, upgrades and failover painful.

Prompts change system behaviour the same way code does, but they are often edited in a text box with no review, history or testing. That is how regressions reach production unnoticed.

I would productise prompt management as a registry. Every prompt has versions with author and timestamp, diffs between versions, and separate environments such as development, staging and production. Each version has linked evaluation results so teams see the quality impact before promoting it. There is one-click rollback, and access controls determine who can edit and who can deploy.

Teams also need tracing: when output quality drops, they should be able to see which prompt version, model and parameters produced a given response.

This lets teams ship prompt changes with the same confidence as code changes. Editing prompts directly in production with no history is the pattern I would want to eliminate.

Defaults should be safe, because most teams will not change them. The baseline set I would provide includes input and output filtering for harmful or policy-violating content, PII detection and masking, defences against prompt injection, controls for topics and policies that a team can configure, rate limits to prevent abuse and runaway cost, and full logging.

Teams can tune thresholds for their context, since a children's education product and an internal legal tool need different settings. But changes are made through a controlled process, logged, and in high-risk cases reviewed by security or the responsible-AI owner.

Guardrail hits must be visible in monitoring. If we never look at what the guardrails block, we cannot tell whether they are too strict, too loose or being bypassed.

Optional guardrails are the failure mode I avoid. If safety needs to be switched on, some teams will forget to do it.

First I protect everyone else, then I fix the underlying cause. In the short term, I apply per-tenant quotas and burst controls so one customer cannot consume shared capacity. Priority queues keep latency-sensitive traffic ahead of batch work.

Clients need to be able to cooperate, so I return clear rate-limit headers and informative error responses that tell them when they can retry. Well-behaved clients then back off automatically and the platform is not overwhelmed by retry storms.

Then I look at the real demand. If this customer has a legitimate high-volume need, I offer a higher tier, reserved capacity or dedicated deployment, priced to cover the cost. I also check whether better practices on their side, like caching or batching, would reduce their load.

I communicate with the customer directly and early, so a limit is a conversation about their needs, not a surprise failure.

I specify data infrastructure as a product requirement, not an afterthought, because model quality is bounded by data quality.

The elements are: reliable ingestion with monitoring so I know when feeds break; automated quality checks for completeness, duplicates and schema changes; lineage so we can trace a model's behaviour back to its data; labelling workflows with clear guidelines and quality review; versioned datasets so experiments are reproducible; and a feature store where several models share features.

I define freshness requirements, such as how stale data may be before it harms predictions, along with access controls and retention rules agreed with security and legal.

The most important line in the spec is ownership: who owns data quality for each source. Without a named owner, quality problems get noticed by users first. I also make sure there is a clear path for production feedback to flow back into training and evaluation data.

I compare options on a consistent set of criteria: functional fit for our actual workload, time to value, scale and performance at our expected volume, security and compliance, lock-in and exit cost, and total cost of ownership including engineering time, hosting and ongoing maintenance.

I prototype with the top two or three candidates on our own data and queries, because vendor demos rarely reflect real conditions. That usually reveals differences in retrieval quality, latency or operational burden that a feature table hides.

I choose to build only if the capability is core to our differentiation, or if existing tools cannot meet a hard requirement such as data residency or scale. Building because it is technically interesting is a trap, since we then own maintenance, upgrades and on-call forever.

I also plan the exit. Even if we buy, I make sure data can be exported and interfaces are abstracted so switching later is feasible.

I make compatibility a design principle. Semantic versioning communicates the impact of a release, and I make changes additive by default: new optional parameters and new endpoints rather than altering what exists.

When something must be deprecated, I issue warnings in the SDK and API responses, publish a timeline and provide a migration guide with examples. Compatibility tests in CI run old client behaviour against new releases so we catch accidental breaks before customers do.

I rely on data: telemetry on which SDK and API versions are in use, and by whom, so I can reach affected customers directly and decide when removal is safe. I will not remove something while meaningful usage remains without a supported path.

Customers build businesses on these interfaces, so removing features without notice, or changing behaviour silently, turns a technical decision into a trust problem.

During an incident, silence is the worst response. I make sure we acknowledge quickly on the status page, even before we know the cause, with a short statement of what is affected.

Then I give honest updates at a predictable cadence, such as every 30 minutes, covering current impact, what we are doing and when the next update will come. I avoid speculation and optimistic estimates I cannot support. One person owns communication so that customers, support and executives all get a consistent message, while engineers stay focused on the fix.

After resolution I send a clear summary and run a blameless review focused on systems and process, not individuals. I publish the findings with actions, owners and dates, so customers see that we learned something.

I never blame vendors publicly, even when they were the trigger. Customers chose us, so we own the reliability of the service they bought.

Agents that call tools can take real actions, so the platform has to give developers power with strong controls.

The core requirements: secure tool registration, where tools are declared with schemas and owners; scoped permissions, so an agent can only use the tools and data it needs and no more; timeouts and retries to handle failures; limits on spend, number of actions and loop depth to prevent runaway behaviour; full trace logging of every step, tool call and result; and human approval hooks for risky actions such as payments, deletions or external messages.

Developers also need a testing sandbox where agents can run against fake or read-only versions of tools, plus evaluation support for multi-step behaviour.

I decide which actions need approval by combining impact and reversibility: anything irreversible, financial or customer-facing should require a human until measured reliability justifies loosening it. Unrestricted tool access with no traces is the pattern that causes serious incidents.

Metering is the foundation of trust and of revenue, so I design it carefully. I meter by tokens or requests, depending on the product, and break usage down by model, tenant, feature and project so customers can see where their spend comes from.

Customers get near-real-time dashboards, configurable budgets and alerts, so no one is surprised by an invoice. Hard caps and soft warnings are both available.

Billing must be transparent and reconcilable: every line on an invoice maps to usage records the customer can export and verify. Accuracy matters, so I build automated reconciliation between metering, logs and billing and investigate any drift immediately.

For a disputed bill, I use the usage logs as the shared source of truth, investigate promptly, and credit quickly where we were wrong. A fast, fair resolution protects the relationship more than winning an argument.

I would not promise anything before I check the facts. First I clarify the actual requirement: is it a legal obligation under Indian regulation, a sector rule such as for financial or health data, a contractual demand from the customer, or a preference? The answer changes what we must deliver.

Then I assess the technical and commercial options: whether our cloud and model providers support hosting in India, which models are available in-region, the cost and latency effects, and what operational changes are needed.

If it is feasible, I offer a region-pinned deployment with clear data-flow documentation showing where data is processed and stored. I confirm that logs, backups, monitoring data and any support access follow the same rule, because those are commonly forgotten and can quietly break the commitment.

I document the architecture, the sub-processors involved and the controls, so the customer's security team can verify it. If we cannot meet it yet, I say so and give an honest timeline.

I treat capacity as a scarce resource to allocate on purpose. First I rank workloads by business value and by latency sensitivity: customer-facing real-time features come first, internal experiments and batch jobs later.

Then I apply mechanisms: quotas per team, scheduling that pushes batch and training jobs to off-peak hours, and priority queues. I also reduce demand through engineering: quantisation, caching, smaller models where quality allows and better batching.

Capacity planning is done with finance and infrastructure teams well ahead of need, since GPU procurement has long lead times. I share a forecast and the assumptions behind it, and I review it regularly as demand changes.

When I have to tell a team they wait, I do it openly: here is the ranking, the reason, and the date we expect capacity. Transparency makes the decision easier to accept than first-come, first-served, which rewards whoever asks first and not whoever matters most.

I would require a model card with every release, because teams cannot use a model responsibly if they do not know what it is and where it fails.

The contents: the purpose and intended use; a summary of the training data and its known gaps; evaluation results broken down by segment, so people see where it is strong and weak; known limitations and failure modes; prohibited or discouraged uses; cost and latency characteristics; the version and its change history; and a contact or channel for reporting issues.

It should be short enough to read and honest enough to be useful. A card that only lists strengths gives teams a false sense of safety.

If I had to pick the most important section, it would be known limitations and intended use, because that is what prevents misuse. I would keep the card updated with each version and link it from the registry so it is found at the point of decision.