RetryAttribute to support decoration of class and/or assembly

[abatishchev]

Summary

Currently, [RetryAttribute] doesn't support decoration of a class or an assembly. This feature request aim to overcome such a limitation.

Background and Motivation

Currently, the attribute is defined as of version 4.0.2:

[AttributeUsage(
    AttributeTargets.Method, AllowMultiple=false,
    Inherited=false)]
public sealed class RetryAttribute : RetryBaseAttribute

This results in a test method like this:

[TestClass]
public class MyEndToEndIntegrationTests
{
    [TestMethod, Retry(3)]
    public void TestA() {}

    [TestMethod, Retry(3)]
    public void TestB() {}

    [TestMethod, Retry(3)]
    public void TestC() {}
}

Proposed Feature

Instead, the attribute would be defined as this:

[AttributeUsage(
    AttributeTargets.Method | AttributeTargets.Class,
    AllowMultiple=false, Inherited=false)]
public sealed class RetryAttribute : RetryBaseAttribute

Or even this:

[AttributeUsage(
    AttributeTargets.Method | AttributeTargets.Class | AttributeTargets.Assembly,
    AllowMultiple=false, Inherited=false)]
public sealed class RetryAttribute : RetryBaseAttribute

Resulting in a test class like this:

[TestClass, Retry(3)]
public class MyEndToEndIntegrationTests
{
    [TestMethod]
    public void TestA() {}

    [TestMethod]
    public void TestB() {}

    [TestMethod]
    public void TestC() {}
}

Or even this:

[assembly: Retry(3)]

Aspects to consider:

• Can a test method override (make it smaller or greater) the value set at a higher level (class, assembly)? If yes, what would be the behavior (permit, ignore, throw)?

Alternative Designs

N/D

[Youssef1313]

For assembly-level retry, if you are using Microsoft.Testing.Platform, you can use Microsoft.Testing.Extensions.Retry for that, which is a platform feature rather than a framework feature. Would that maybe serve your purpose better?

[abatishchev]

In order to leverage Testing Platform, I need to migrate my tests (ideally and eventually, all service's repo, which would be a huge undertaking), correct? And that's a problem. I tried doing that earlier (twice, even), but since it's generates a self-contained app, it's not compatible with referencing another self-contained app (which is my service itself).

The service is built as a self-contained app for reasons unrelated how it's tested but how it's deployed. So instead of relying on the version of .NET SDK available on the hosting platform (and hence, be up to their mercy), it controls the version at the build time.

In another words, the migration to Testing Platform continues to be blocked for me.

[Evangelink]

but since it's generates a self-contained app, it's not compatible with referencing another self-contained app (which is my service itself).

That's pretty surprising. Could you create a small repro of the issue for us to investigate?

[Youssef1313]

@abatishchev Are you referring to NETSDK1151 warning which happens when a non-SelfContained app references a SelfContained app? You should be able to build both as SelfContained. Basically, with MTP you have more freedom to build with the model the suits your needs, while with VSTest, you're forced to load the test assembly into however testhost is shipped.

I also see this curious suppressed warning in VSTest's testhost:

https://github.com/microsoft/vstest/blob/6c9778ead1d80ed0d8212314e83d04e92443f1f8/src/testhost.x86/testhost.x86.csproj#L28

Looks like this was warning about a breaking change in .NET 8 that VSTest might haven't addressed the right way (possibly it was previously shipping SelfContained testhost but now no longer? I'm not sure - need to take a closer look at that, but that's pretty much a different issue - just brought it as it's closely related)

[abatishchev]

I googled the issue and the error (by memory), and it might be NETSDK1150 or NETSDK1151, but that a build error, not a warning. So I have up (I tried twice). But now I see it can be disabled by:

<ValidateExecutableReferencesMatchSelfContained>false</ValidateExecutableReferencesMatchSelfContained>

And then the AI suggests also:

<ProjectReference Include="{PathToReferencedProject}.csproj" GlobalPropertiesToRemove="SelfContained" />

Let me revive the PR and I'll provide the concrete error and try to come up with a small repro.

[abatishchev]

Interestingly, I'm (at least, was) not using VSTest. Only MSTest and DotNetCli@2 with command=test. But then my team had to migrate CloudTest (and honestly, I lost track what we do and what we don't).