Which rating would apply to a result that contains broad travel information but fails to address the user's concrete question about a flight?

Which rating would apply to a result that contains broad travel information but fails to address the user's concrete question about a flight?

• A. FM
• B. FailsM
• C. SM
• D. Not Relevant

Answer: (A)

When evaluating usefulness, we look at how well a result meets the user’s exact need, not just whether it has some related content. Here, the user asks for flight-specific information, but the result provides broad travel info and doesn’t answer the flight question. That means it isn’t fully met, but it isn’t irrelevant either since there is some travel context that could be useful. The best rating for this mix—information present but not addressing the concrete request—is FM. It acknowledges partial usefulness while signaling the key shortcoming: the specific flight question isn’t addressed.

When evaluating usefulness, we look at how well a result meets the user’s exact need, not just whether it has some related content. Here, the user asks for flight-specific information, but the result provides broad travel info and doesn’t answer the flight question. That means it isn’t fully met, but it isn’t irrelevant either since there is some travel context that could be useful. The best rating for this mix—information present but not addressing the concrete request—is FM. It acknowledges partial usefulness while signaling the key shortcoming: the specific flight question isn’t addressed.