Serenity 6.9.3 Release Notes (2023-10-22)
New GenerateFieldsAttribute for Serenity.Pro.Coder
In Serenity.Pro.Coder 6.1.0, we introduced the RowTemplate mechanism, which enables you to define classes as shown below:
public partial class SimpleRow
{
private class RowTemplate
{
[SomeAttribute]
public string Name { get; set; }
public int? Age { get; set; }
}
}
Subsequently, the source generator generates a partial class with corresponding properties and field definitions for you, as follows:
using Serenity.Data;
partial class SimpleRow : Row<SimpleRow.RowFields>
{
private string name;
private int? age;
[SomeAttribute]
public string Name { get => fields.Name[this]; set => fields.Name[this] = value; }
public int? Age { get => fields.Age[this]; set => fields.Age[this] = value; }
public partial class RowFields : RowFieldsBase
{
public StringField Name;
public Int32Field Age;
protected override void CreateGeneratedFields()
{
Name = //...
Age = //...
}
}
}
This feature simplifies the process of defining row classes and adding new properties to them later. However, as it started to be used more extensively, some peculiarities became apparent.
For instance, because the attributes are defined in the template class rather than the row properties, the source generator had to duplicate these attributes to the row properties it generated. This seemingly straightforward task is, in fact, not trivial. The source generator must analyze the attribute classes, identify their namespaces to include them in the using list, and handle any constructor parameters, which can be external types in addition to primitive ones like int and string.
Another issue arises when some attributes are generated by other source generators, such as editor type attributes created by the client types source generator. Since source generators cannot access each other's output, we had to manually execute these other source generators that may generate attributes. While we can do this for our own generators, it raises concerns about unknown source generators that might also create attributes.
To address these challenges, we have developed a new type of fields source generator using the [GenerateFields] attribute:
[GenerateFields]
public partial class SimpleRow
{
[SomeAttribute]
public string Name { get => fields.Name[this]; set => fields.Name[this] = value; }
public int? Age { get => fields.Age[this]; set => fields.Age[this] = value; }
}
With this approach, the generated partial class becomes:
using Serenity.Data;
partial class SimpleRow : Row<SimpleRow.RowFields>
{
private string name;
private int? age;
public partial class RowFields : RowFieldsBase
{
public StringField Name;
public Int32Field Age;
protected override void CreateGeneratedFields()
{
Name = //...
Age = //...
}
}
}
While the end result appears similar, with the [GenerateFields] source generator, you only need to write getters and setters as you normally would. If C# supported partial properties, the source generator could handle this for you, but this feature is still under consideration.
By using the [GenerateFields] source generator, you eliminate the need to create properties and copy their attributes, avoiding the issues associated with the RowTemplate mechanism. Additionally, other source generators can easily analyze the row class without having to execute the row fields source generator, as the properties are already defined in the user-written SimpleRow class.
This is now the recommended approach, as it is less error-prone and more efficient. In the future, when C# introduces partial properties, you will no longer need to write getter and setter code as well.
IRolePermissionService
In Serenity, an IPermissionService interface is provided for verifying if the current user possesses a specific permission:
public interface IPermissionService
{
bool HasPermission(string permission);
}
The implementation in StartSharp/Serene includes the logic to verify both the permissions directly assigned to the current user and those indirectly assigned via their roles. This crucial information is encapsulated within the IPermissionService implementation and cannot be easily reused in other contexts. To address this, we have introduced a new interface dedicated to checking permissions based on user roles:
public interface IRolePermissionService
{
bool HasPermission(string role, string permission);
}
The role argument here can either be the role key, if available, or the role name if the key is not present.
This interface is designed to be injectable into other services, enabling role-based permission checks. Furthermore, the implementations in StartSharp/Serene have been adjusted to incorporate this interface, ensuring seamless integration and expanded functionality.
OpenId Scopes to Permission Set Mapping
We recently introduced the integration of OpenIddict via the Serenity.Pro.OpenIddict feature package. This integration empowers you to grant internal and external applications access to your account with ease. With OpenIdConnect and your consent, these applications can securely access your account, opening up a realm of possibilities. For instance, Serenity demo application seamlessly access your entries in the Work Log module.
However, there is a critical issue that arises when applications are not configured properly. If left unchecked, such applications can potentially call any API or endpoint with AllowCookieAndOAuth or AllowOnlyOauth attributes
via JWT/OAuth on behalf of the user.
For instance, let's say you wish to grant access to both the Contacts and Work Log services, which are intended for use by different applications. In such cases, it becomes essential to limit application permissions so that granting access to one service does not inadvertently open doors to access other data or services.
This limitation is typically implemented through the use of scopes. Scopes play a pivotal role in managing and restricting application permissions. Even though a user might possess certain permissions for a particular module, the application itself does not automatically inherit these permissions. To simplify usability, we have established a systematic mapping between scopes and permissions.
To map scopes to permissions, you can make adjustments in your appsettings.json:
{
"OpenIdSettings": {
//...
"Scopes": [
{ "Name": "profile" },
{ "Name": "email" },
{ "Name": "worklog", "PermissionFilters": ["Regex:^WorkLog:.*"] },
{ "Name": "northwind", "PermissionFilters": ["Role:NorthwindAdmin"] },
{ "Name": "northwind_general", "PermissionFilters": ["Northwind:General"] }
]
}
}
In the provided example, an application with the worklog scope is granted permissions that match any starting with WorkLog: based on a Regex filter. This secondary level filter ensures that the application can only access a subset of the user's permissions. If the user does not possess a specific permission, such as WorkLog:Administrator, the application will not have it either.
The northwind scope, on the other hand, utilizes a permission filter defined as Role:NorthwindAdmin. This filter matches the permissions assigned to a role with the key or name NorthwindAdmin. This approach allows you to create a role in the Roles screen, assign it specific permissions, and use it as a filter to limit scope permissions. Remarkably, this role need not be assigned to any user, affording the flexibility to change scope permissions at runtime without editing appsettings.json or restarting the application.
Additionally, you can directly list permission keys for scopes, such as in the northwind_general scope.
To make this feature work seamlessly, you must assign these scopes to specific applications in your appsettings.json:
"OpenIdSettings": {
"Applications": [{
"ClientId": "worklogapp",
"Permissions": [
// other scopes
"scp:worklog"
]
}, {
"ClientId": "somenorthwindapp",
"Permissions": [
// other scopes
"scp:northwind",
"scp:northwind_general"
]
}]
}
For proper functionality, be sure to use the OpenIdScopePermissionWrapper to wrap the IPermissionService implementation in your project by registering it in Startup.cs:
services.AddSingleton<IPermissionService>(s =>
ActivatorUtilities.CreateInstance<Serenity.Pro.OpenIddict.OpenIdScopePermissionWrapper>(s,
ActivatorUtilities.CreateInstance<AppServices.PermissionService>(s)));
If you are employing the LogicOperatorPermissionService wrapper, your registration should resemble the following:
services.AddSingleton<IPermissionService>(s =>
ActivatorUtilities.CreateInstance<LogicOperatorPermissionService(s,
ActivatorUtilities.CreateInstance<Serenity.Pro.OpenIddict.OpenIdScopePermissionWrapper>(s,
ActivatorUtilities.CreateInstance<Administration.PermissionService>(s))));
It's worth noting that as the OpenIdScopePermissionWrapper requires access to role permissions, an IRolePermissionService implementation should also be registered, which is typically done in any StartSharp/Serene application version later than 6.8.1:
services.AddSingleton<Serenity.Net.Core.Authorization.IRolePermissionService, AppServices.RolePermissionService>();
We trust that this powerful addition will further enhance the security and functionality of your Serenity-based applications.
New Application Specific TypeSource Implementation
In an effort to streamline and enhance the application discovery process in Serenity, we have introduced a new implementation for the ITypeSource interface. This interface is pivotal in Serenity as it relies on reflection to discover types within your application. In previous versions, it was a common practice to create a DefaultTypeSource instance in your Startup.cs and populate it with the assemblies of your application's feature sets. Here's how it used to look:
services.AddSingleton<ITypeSource>(new DefaultTypeSource(new[]
{
typeof(LocalTextRegistry).Assembly,
//...
typeof(Serenity.Pro.Extensions.BackgroundJobManager).Assembly,
typeof(Serenity.Demo.Northwind.CustomerPage).Assembly,
//..
}
Notably, if an assembly representing a feature is not included in this list, its types, including entities, forms, columns, lookup scripts, and migrations, would remain undiscovered by Serenity. Therefore, it was crucial to ensure that all feature packages were explicitly added.
With this update, we have relocated the declaration of the type source to a dedicated file located at Initialization/TypeSource.cs. This file contains a new class, TypeSource, which inherits from DefaultTypeSource. Here's what it looks like:
namespace StartSharp.AppServices;
public class TypeSource : DefaultTypeSource
{
public TypeSource()
: base(GetAssemblyList())
{
}
private static Assembly[] GetAssemblyList()
{
return new Assembly[]
{
typeof(LocalTextRegistry).Assembly,
//...
typeof(Serenity.Pro.Extensions.BackgroundJobManager).Assembly,
typeof(Serenity.Demo.Northwind.CustomerPage).Assembly,
//...
}
}
}
With this approach, the registration of the type source in your Startup.cs becomes much cleaner:
var typeSource = new AppServices.TypeSource();
services.AddSingleton<ITypeSource>(typeSource);
New ConfigureSections Extension
In a previous version of Serenity, we introduced the ConfigureSection extension method to facilitate the binding of option classes to their default configuration sections. This method required explicit registration in your Startup.cs and looked something like this:
services.ConfigureSection<BackgroundJobSettings>(Configuration);
services.ConfigureSection<ClamAVSettings>(Configuration);
services.ConfigureSection<ConnectionStringOptions>(Configuration);
//...
services.ConfigureSection<Serenity.Pro.DataExplorer.DataExplorerConfig>(Configuration);
While this approach simplified configuration, it was not entirely elegant, especially when adding new option classes. Any new options type introduced would necessitate additional manual configuration in your Startup.cs.
To address this, we have harnessed the power of the ITypeSource service and introduced the new ConfigureSections extension. This extension has the potential to significantly reduce the amount of configuration code in your Startup.cs. Here's how it works:
services.ConfigureSections(Configuration, typeSource);
With this single line of code, Serenity will scan all assemblies within the type source and automatically invoke the ConfigureSection method for every option class adorned with the [DefaultSectionKey] attribute.
As a result, adding a new option class no longer requires any manual changes to your Startup.cs. This improvement not only simplifies your configuration but also ensures that your application remains flexible and adaptable as new options are introduced.
File Scoped Namespaces and Global Usings in Serene / StartSharp
Serene and StartSharp now uses C#'s file scoped namespaces feature. Instead of:
namespace SomeNamespace
{
public class SomeClass
{
//...
}
}
We now use:
namespace SomeNamespace;
public class SomeClass
{
//...
}
This looks better in our opinion as it reduces one level of indentation / braces.
We also made use of global usings feature, by including a set of namespaces by default in the project file:
<Using Include="Microsoft.AspNetCore.Mvc;Microsoft.Extensions.Options;System;System.Collections.Generic;System.ComponentModel;System.Linq;System.Text;System.Threading" />
<Using Include="Serenity;Serenity.Abstractions;Serenity.ComponentModel;Serenity.Data;Serenity.Data.Mapping;Serenity.Extensions;Serenity.Pro.Extensions;Serenity.Services;Serenity.Web" />
<Using Include="System.Data.IDbConnection" Alias="IDbConnection" />
This reduces the amount of using statement in source files. The set of namespaces are chosen carefully to include mostly used ones in Serenity applications, while reducing the possibly for any type name clashes.
Please note that Sergen can't yet parse the list of global usings, so the files it generates might still have them in the using list. To workaround that for now, you may manually list them in sergen.json:
{
//...
"IncludeGlobalUsings": [
"Serenity",
"Serenity.ComponentModel",
"Serenity.Extensions",
"System",
"System.Collections.Generic",
"System.ComponentModel"
]
}
Migration Related Changes
Using Tags Instead of Namespaces
In the realm of migration management in Serenity, we've made a significant change to streamline and enhance the migration process. In the past, migration classes were organized within specific namespaces to determine their execution on a particular database. For example:
namespace StartSharp.Migrations.DefaultDB;
[Migration(20141103_1400)]
public class DefaultDB_20141103_1400_Initial : AutoReversingMigration
{
}
The namespace of the migrations played a crucial role, as it was used in DataMigrations.cs to determine which migrations should be executed by the FluentMigrator Runner on a specific database. If a migration was placed in a different namespace by mistake or if users were not aware of this rule, the migration would not be executed.
Another challenge arose when dealing with feature packages like DataAuditLog or WorkLog. Since there was no way to match their namespaces with the application's requirements, we had to place migrations for feature packages directly in the StartSharp or Serene templates. When adding a feature package to an existing application, manual copying of the required migrations was necessary.
To address these issues, we've modified the logic in DataMigrations.cs to utilize FluentMigrator's Tags attribute instead of relying on namespaces. Now, migrations can be structured as follows:
namespace NamespaceHere.IsNoLongerRelevant;
[Tags("DefaultDB"), Migration(20141103_1400)]
public class DefaultDB_20141103_1400_Initial : AutoReversingMigration
{
}
To further reduce the chance of typographical errors, a DefaultDB attribute is now available in Serenity.Extensions, which can be used in place of Tags:
namespace NamespaceHere.IsNoLongerRelevant;
[DefaultDB, Migration(20141103_1400)]
public class DefaultDB_20141103_1400_Initial : AutoReversingMigration
{
}
For backward compatibility, the existing implementation in DataMigrations.cs is configured to run migrations without any tags. This ensures that existing migrations in your project that lack the "DefaultDB" tag will still run on the Default database:
options.Tags = new[] { databaseKey + "DB" };
options.IncludeUntaggedMigrations = databaseKey == "Default";
All migrations are now relocated to their respective feature packages. As a result, after upgrading to version 6.9.2+, you can safely delete the following migrations from your project (if you haven't modified them):
- DefaultDB_20161126_2000_Organization
- DefaultDB_20161126_2100_Meeting
- DefaultDB_20170211_1527_Mailing
- DefaultDB_20180513_2014_DataAuditLog
- DefaultDB_20210522_1623_MailSerializedMessage
- DefaultDB_20210824_1546_WorkLog
- DefaultDB_20220629_1515_OpenIddict
This means that if a feature package is part of the type source, its migrations will automatically run on the Default database. You can no longer simply remove it from your application as was the case previously.
If there is a need to disable specific migrations while retaining the feature package in the type source, you will need to filter out those assemblies in the following line in DataMigrations.cs:
builder.ScanIn(((IGetAssemblies)typeSource).GetAssemblies().ToArray()).For.Migrations();
For instance, to exclude migrations in Serenity.Pro.DataAuditLog, you can modify the code like this:
builder.ScanIn(((IGetAssemblies)typeSource).GetAssemblies()
.Where(x => x.GetName().Name != "Serenity.Pro.DataAuditLog")
.ToArray()).For.Migrations();
New MigrationKey Attribute
In the past, we utilized a custom Migrations/MigrationAttribute.cs file in your project, which contained logic to validate migration keys, ensuring they followed the format yyyyMMdd_HHmmss (or yyyyMMdd_HHmm) and maintained a consistent number of digits for migration running order.
With this update, we've migrated this logic to the Serenity.Extensions namespace as the MigrationKey attribute. This means you can now delete the custom MigrationAttribute.cs file from your project and replace all [Migration] attributes with [MigrationKey].
namespace NamespaceHere.IsNoLongerRelevant;
[DefaultDB, MigrationKey(20141103_1400)]
public class DefaultDB_20141103_1400_Initial : AutoReversingMigration
{
}
It's important to replace all [Migration] attributes with [MigrationKey] attributes, as failure to do so may result in issues. When a version is specified in the yyyyMMdd_HHmm format, it is mapped to yyyyMMdd_HHmm00 by the logic in both the custom Migration attribute and the new MigrationKey attribute. However, FluentMigrator's own [Migration] attribute does not apply this logic, and as a result, your migrations may attempt to run again, causing errors.
Removed the Database Name Check Before Executing Migrations
Previously, in DataMigrations.cs, there was a check in place that prevented Serenity from running migrations on a database with a name that did not match the expected pattern. For example, if your project was named MyProject, the database name in the connection string had to be MyProject_Default_v1 for the migrations to proceed. This check was intended to safeguard against unintentional data loss or damage when migrating to a production database. If, for instance, a migration deleted a column or table, it could have serious consequences in a production environment.
To address this concern, developers were advised to remove the check after understanding the associated risks. However, this approach seemed to create more confusion for newcomers rather than providing clear benefits. Despite our efforts to indicate that migrations were being skipped due to a database name mismatch on the login page, it led to frustration and misunderstanding.
After careful consideration, we have decided to remove this check. It is now the responsibility of developers to exercise caution while writing migrations to ensure that they do not inadvertently damage data. This change aims to simplify the migration process and provide more flexibility, though it underscores the importance of being vigilant when making database changes through migrations.
Sergen Entity Code Generation Enhancements
When generating code for a new table using dotnet sergen g, Sergen previously did not consider existing Row classes in your application. For example, if you were generating code for a new EmployeeTasks table with a foreign key column EmployeeId, and you already had an EmployeeRow defined in your project, like this:
[LookupScript]
public sealed class EmployeeRow : Row<EmployeeRow.RowFields>, IIdRow, INameRow
{
[DisplayName("Employee Id"), Identity, IdProperty]
public int? EmployeeId { get => fields.EmployeeId[this]; set => fields.EmployeeId[this] = value; }
[DisplayName("Last Name"), Size(20), NotNull]
public string LastName { get => fields.LastName[this]; set => fields.LastName[this] = value; }
[DisplayName("First Name"), Size(10), NotNull]
public string FirstName { get => fields.FirstName[this]; set => fields.FirstName[this] = value; }
[DisplayName("FullName"), NameProperty, QuickSearch]
[Concat($"T0.[{nameof(FirstName)}]", "' '", $"T0.[{nameof(LastName)}")]
public string FullName { get => fields.FullName[this]; set => fields.FullName[this] = value; }
//...
}
The EmployeeRow entity had a FullName expression field defined as the NameProperty of the entity.
If Sergen generated code for EmployeeTasks, it previously looked like this:
public sealed class EmployeeTasksRow : Row<EmployeeTasksRow.RowFields>
{
const string jEmployee = nameof(jEmployee);
[DisplayName("Employee Task Id"), Identity, IdProperty]
public int? EmployeeTaskId { get => fields.EmployeeTaskId[this]; set => fields.EmployeeTaskId[this] = value; }
[DisplayName("Employee Id"), ForeignKey("Employees", "EmployeeId"), LeftJoin(jEmployee), TextualField(nameof(EmployeeLastName))]
public int? EmployeeId { get => fields.EmployeeId[this]; set => fields.EmployeeId[this] = value; }
[DisplayName("Employee Last Name"), Expression($"{jEmployee}.LastName")]
public string EmployeeLastName { get => fields.EmployeeLastName[this]; set => fields.EmployeeLastName[this]; }
}
In this case, Sergen identified the foreign key and added a [ForeignKey] attribute to the EmployeeId property. However, it used string references instead of referencing the EmployeeRow type directly. Additionally, even though there was a FullName property in EmployeeRow, Sergen assumed that the LastName column was the name property of the employee since it wasn't aware of existing entities in your project during code generation.
In the new version, Sergen generates code like this:
[ServiceLookupPermission("SomePermission")]
public sealed class EmployeeTasksRow : Row<EmployeeTasksRow.RowFields>
{
const string jEmployee = nameof(jEmployee);
[DisplayName("Employee Task Id"), Identity, IdProperty]
public int? EmployeeTaskId { get => fields.EmployeeTaskId[this]; set => fields.EmployeeTaskId[this] = value; }
[DisplayName("Employee Id"), ForeignKey(typeof(EmployeeRow)), LeftJoin(jEmployee), TextualField(nameof(EmployeeFullName))
[LookupEditor]
public int? EmployeeId { get => fields.EmployeeId[this]; set => fields.EmployeeId[this]; }
[DisplayName("Employee Full Name"), Origin(jEmployee, nameof(EmployeeRow.FullName))
public string EmployeeFullName { get => fields.EmployeeFullName[this]; set => fields.EmployeeFullName[this]; }
}
In this updated code, Sergen correctly uses typeof(EmployeeRow) for the ForeignKey attribute, identifies the name property as FullName, defines the textual field as EmployeeFullName, and uses the Origin attribute instead of the Expression attribute. This allows the expression of EmployeeFullName to be derived from the definition in EmployeeRow.FullName property.
Please note that for this feature to work, your project must be built successfully before running dotnet sergen g, as Sergen scans your output assemblies for existing entity types.
Sergen also adds a [LookupEditor] attribute to the EmployeeId property if it detects that EmployeeRow already has a [LookupScript] attribute. If not, Sergen adds a [ServiceLookupEditor] attribute.
Sergen also includes a ServiceLookupPermission in newly generated entities, making it easy to adjust permissions for accessing lookup columns.
When generating code for multiple tables at once, Sergen can now also reuse information for those entities while generating code for each one, treating them as if they are already available as Row classes in your project.
Column Typings Provide Easy Access to Columns by Property Name
Previously, there were code blocks like this in Grid.ts files:
protected getColumns() {
var columns = super.getColumns();
var impersonate = tryFirst(columns, x => x.field == "ImpersonationToken");
if (impersonate != null) {
impersonate.format = ctx => {
//...
return columns;
}
To modify a column, you needed to find it by its field name using the tryFirst helper method.
Now, the Columns.ts files generated by Sergen/Pro.Coder provide convenient properties like this:
//...
export interface UserColumns {
UserId: Column<UserRow>;
Username: Column<UserRow>;
ImpersonationToken: Column<UserRow>;
DisplayName: Column<UserRow>;
Email: Column<UserRow>;
Source: Column<UserRow>;
Roles: Column<UserRow>;
}
export class UserColumns extends ColumnsBase<UserRow> {
static readonly columnsKey = 'Administration.User';
static readonly Fields = fieldsProxy<UserColumns>();
}
This structure allows you to modify columns as follows:
protected getColumns() {
var columns = new UserColumns(super.getColumns());
columns.ImpersonationToken && (columns.ImpersonationToken.format = ctx => {
//...
return columns.valueOf();
}
This enhancement not only helps to avoid typing mistakes but also makes the process faster since there's no need to search for every column in the array. Additionally, it provides code completion and compile-time checking.