برچسب: Logging

  • Easy logging management with Seq and ILogger in ASP.NET | Code4IT

    Easy logging management with Seq and ILogger in ASP.NET | Code4IT


    Seq is one of the best Log Sinks out there : it’s easy to install and configure, and can be added to an ASP.NET application with just a line of code.

    Table of Contents

    Just a second! 🫷
    If you are here, it means that you are a software developer.
    So, you know that storage, networking, and domain management have a cost .

    If you want to support this blog, please ensure that you have disabled the adblocker for this site.
    I configured Google AdSense to show as few ADS as possible – I don’t want to bother you with lots of ads, but I still need to add some to pay for the resources for my site.

    Thank you for your understanding.
    Davide

    Logging is one of the most essential parts of any application.

    Wouldn’t it be great if we could scaffold and use a logging platform with just a few lines of code?

    In this article, we are going to learn how to install and use Seq as a destination for our logs, and how to make an ASP.NET 8 API application send its logs to Seq by using the native logging implementation.

    Seq: a sink and dashboard to manage your logs

    In the context of logging management, a “sink” is a receiver of the logs generated by one or many applications; it can be a cloud-based system, but it’s not mandatory: even a file on your local file system can be considered a sink.

    Seq is a Sink, and works by exposing a server that stores logs and events generated by an application. Clearly, other than just storing the logs, Seq allows you to view them, access their details, perform queries over the collection of logs, and much more.

    It’s free to use for individual usage, and comes with several pricing plans, depending on the usage and the size of the team.

    Let’s start small and install the free version.

    We have two options:

    1. Download it locally, using an installer (here’s the download page);
    2. Use Docker: pull the datalust/seq image locally and run the container on your Docker engine.

    Both ways will give you the same result.

    However, if you already have experience with Docker, I suggest you use the second approach.

    Once you have Docker installed and running locally, open a terminal.

    First, you have to pull the Seq image locally (I know, it’s not mandatory, but I prefer doing it in a separate step):

    Then, when you have it downloaded, you can start a new instance of Seq locally, exposing the UI on a specific port.

    docker run --name seq -d --restart unless-stopped -e ACCEPT_EULA=Y -p 5341:80 datalust/seq:latest
    

    Let’s break down the previous command:

    • docker run: This command is used to create and start a new Docker container.
    • --name seq: This option assigns the name seq to the container. Naming containers can make them easier to manage.
    • -d: This flag runs the container in detached mode, meaning it runs in the background.
    • --restart unless-stopped: This option ensures that the container will always restart unless it is explicitly stopped. This is useful for ensuring that the container remains running even after a reboot or if it crashes.
    • -e ACCEPT_EULA=Y: This sets an environment variable inside the container. In this case, it sets ACCEPT_EULA to Y, which likely indicates that you accept the End User License Agreement (EULA) for the software running in the container.
    • -p 5341:80: This maps port 5341 on your host machine to port 80 in the container. This allows you to access the service running on port 80 inside the container via port 5341 on your host.
    • datalust/seq:latest: This specifies the Docker image to use for the container. datalust/seq is the image name, and latest is the tag, indicating that you want to use the latest version of this image.

    So, this command runs a container named seq in the background, ensures it restarts unless stopped, sets an environment variable to accept the EULA, maps a host port to a container port, and uses the latest version of the datalust/seq image.

    It’s important to pay attention to the used port: by default, Seq uses port 5341 to interact with the UI and the API. If you prefer to use another port, feel free to do that – just remember that you’ll need some additional configuration.

    Now that Seq is installed on your machine, you can access its UI. Guess what? It’s on localhost:5341!

    Seq brand new instance

    However, Seq is “just” a container for our logs – but we have to produce them.

    A sample ASP.NET API project

    I’ve created a simple API project that exposes CRUD operations for a data model stored in memory (we don’t really care about the details).

    [ApiController]
    [Route("[controller]")]
    public class BooksController : ControllerBase
    {
        public BooksController()
        {
    
        }
    
        [HttpGet("{id}")]
        public ActionResult<Book> GetBook([FromRoute] int id)
        {
    
            Book? book = booksCatalogue.SingleOrDefault(x => x.Id == id);
            return book switch
            {
                null => NotFound(),
                _ => Ok(book)
            };
        }
    }
    

    As you can see, the details here are not important.

    Even the Main method is the default one:

    var builder = WebApplication.CreateBuilder(args);
    
    builder.Services.AddControllers();
    
    builder.Services.AddEndpointsApiExplorer();
    builder.Services.AddSwaggerGen();
    
    var app = builder.Build();
    
    if (app.Environment.IsDevelopment())
    {
        app.UseSwagger();
        app.UseSwaggerUI();
    }
    
    app.UseHttpsRedirection();
    
    app.MapControllers();
    
    app.Run();
    

    We have the Controllers, we have Swagger… well, nothing fancy.

    Let’s mix it all together.

    How to integrate Seq with an ASP.NET application

    If you want to use Seq in an ASP.NET application (may it be an API application or whatever else), you have to add it to the startup pipeline.

    First, you have to install the proper NuGet package: Seq.Extensions.Logging.

    The Seq.Extensions.Logging NuGet package

    Then, you have to add it to your Services, calling the AddSeq() method:

    var builder = WebApplication.CreateBuilder(args);
    
    builder.Services.AddControllers();
    
    builder.Services.AddEndpointsApiExplorer();
    builder.Services.AddSwaggerGen();
    
    + builder.Services.AddLogging(lb => lb.AddSeq());
    
    var app = builder.Build();
    

    Now, Seq is ready to intercept whatever kind of log arrives at the specified port (remember, in our case, we are using the default one: 5341).

    We can try it out by adding an ILogger to the BooksController constructor:

    private readonly ILogger<BooksController> _logger;
    
    public BooksController(ILogger<BooksController> logger)
    {
        _logger = logger;
    }
    

    So that we can use the _logger instance to create logs as we want, using the necessary Log Level:

    [HttpGet("{id}")]
    public ActionResult<Book> GetBook([FromRoute] int id)
    {
        _logger.LogInformation("I am Information");
        _logger.LogWarning("I am Warning");
        _logger.LogError("I am Error");
        _logger.LogCritical("I am Critical");
    
        Book? book = booksCatalogue.SingleOrDefault(x => x.Id == id);
        return book switch
        {
            null => NotFound(),
            _ => Ok(book)
        };
    }
    

    Log messages on Seq

    Using Structured Logging with ILogger and Seq

    One of the best things about Seq is that it automatically handles Structured Logging.

    [HttpGet("{id}")]
    public ActionResult<Book> GetBook([FromRoute] int id)
    {
        _logger.LogInformation("Looking if in my collection with {TotalBooksCount} books there is one with ID {SearchedId}"
     , booksCatalogue.Count, id);
    
        Book? book = booksCatalogue.SingleOrDefault(x => x.Id == id);
        return book switch
        {
            null => NotFound(),
            _ => Ok(book)
        };
    }
    

    Have a look at this line:

    _logger.LogInformation("Looking if in my collection with {TotalBooksCount} books there is one with ID {SearchedId}"
     , booksCatalogue.Count, id);
    

    This line generates a string message, replaces all the placeholders, and, on top of that, creates two properties, SearchedId and TotalBooksCount; you can now define queries using these values.

    Structured Logs in Seq allow you to view additional logging properties

    Further readings

    I have to admit it: logging management is one of my favourite topics.

    I’ve already written a sort of introduction to Seq in the past, but at that time, I did not use the native ILogger, but Serilog, a well-known logging library that added some more functionalities on top of the native logger.

    🔗 Logging with Serilog and Seq | Code4IT

    This article first appeared on Code4IT 🐧

    In particular, Serilog can be useful for propagating Correlation IDs across multiple services so that you can fetch all the logs generated by a specific operation, even though they belong to separate applications.

    🔗 How to log Correlation IDs in .NET APIs with Serilog

    Feel free to search through my blog all the articles related to logging – I’m sure you will find interesting stuff!

    Wrapping up

    I think Seq is the best tool for local development: it’s easy to download and install, supports structured logging, and can be easily added to an ASP.NET application with just a line of code.

    I usually add it to my private projects, especially when the operations I run are complex enough to require some well-structured log.

    Given how it’s easy to install, sometimes I use it for my work projects too: when I have to fix a bug, but I don’t want to use the centralized logging platform (since it’s quite complex to use), I add Seq as a destination sink, run the application, and analyze the logs in my local machine. Then, of course, I remove its reference, as I want it to be just a discardable piece of configuration.

    I hope you enjoyed this article! Let’s keep in touch on LinkedIn, Twitter or BlueSky! 🤜🤛

    Happy coding!

    🐧





    Source link

  • HTTP Logging in ASP.NET: how to automatically log all incoming HTTP requests (and its downsides!)

    HTTP Logging in ASP.NET: how to automatically log all incoming HTTP requests (and its downsides!)


    Aren’t you tired of adding manual logs to your HTTP APIs to log HTTP requests and responses? By using a built-in middleware in ASP.NET, you will be able to centralize logs management and have a clear view of all the incoming HTTP requests.

    Table of Contents

    Just a second! 🫷
    If you are here, it means that you are a software developer.
    So, you know that storage, networking, and domain management have a cost .

    If you want to support this blog, please ensure that you have disabled the adblocker for this site.
    I configured Google AdSense to show as few ADS as possible – I don’t want to bother you with lots of ads, but I still need to add some to pay for the resources for my site.

    Thank you for your understanding.
    Davide

    Whenever we publish a service, it is important to add proper logging to the application. Logging helps us understand how the system works and behaves, and it’s a fundamental component that allows us to troubleshoot problems that occur during the actual usage of the application.

    In this blog, we have talked several times about logging. However, we mostly focused on the logs that were written manually.

    In this article, we will learn how to log incoming HTTP requests to help us understand how our APIs are being used from the outside.

    Scaffolding the empty project

    To showcase this type of logging, I created an ASP.NET API. It’s a very simple application with CRUD operations on an in-memory collection.

    [ApiController]
    [Route("[controller]")]
    public class BooksController : ControllerBase
    {
        private readonly List<Book> booksCatalogue = Enumerable.Range(1, 5).Select(index => new Book
        {
            Id = index,
            Title = $"Book with ID {index}"
        }).ToList();
    
        private readonly ILogger<BooksController> _logger;
    
        public BooksController(ILogger<BooksController> logger)
        {
            _logger = logger;
        }
    }
    

    These CRUD operations are exposed via HTTP APIs, following the usual verb-based convention.

    For example:

    [HttpGet("{id}")]
    public ActionResult<Book> GetBook([FromRoute] int id)
    {
        _logger.LogInformation("Looking if in my collection with {TotalBooksCount} books there is one with ID {SearchedId}"
                , booksCatalogue.Count, id);
    
        Book? book = booksCatalogue.SingleOrDefault(x => x.Id == id);
    
        return book switch
        {
            null => NotFound(),
            _ => Ok(book)
        };
    }
    

    As you can see, I have added some custom logs: before searching for the element with the specified ID, I also wrote a log message such as “Looking if in my collection with 5 books there is one with ID 2”.

    Where can I find the message? For the sake of this article, I decided to use Seq!

    Seq is a popular log sink (well, as you may know, my favourite one!), that is easy to install and to integrate with .NET. I’ve thoroughly explained how to use Seq in conjunction with ASP.NET in this article and in other ones.

    In short, the most important change in your application is to add Seq as the log sink, like this:

    builder.Services.AddLogging(lb => {
        lb.AddSeq();
    });
    

    Now, whenever I call the GET endpoint, I can see the related log messages appear in Seq:

    Custom log messages

    But sometimes it’s not enough. I want to see more details, and I want them to be applied everywhere!

    How to add HTTP Logging to an ASP.NET application

    HTTP Logging is a way of logging most of the details of the incoming HTTP operations, tracking both the requests and the responses.

    With HTTP Logging, you don’t need to manually write custom logs to access the details of incoming requests: you just need to add its related middleware, configure it as you want, and have all the required logs available for all your endpoints.

    Adding it is pretty straightforward: you first need to add the HttpLogging middleware to the list of services:

    builder.Services.AddHttpLogging(lb => { });
    

    so that you can use it once the WebApplication instance is built:

    There’s still a problem, though: all the logs generated via HttpLogging are, by default, ignored, as logs coming from their namespace (named Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware) are at Information log level, thus ignored because of the default configurations.

    You either have to update the appsetting.json file to tell the logging system to process logs from that namespace:

    {
      "Logging": {
        "LogLevel": {
          "Default": "Information",
          "Microsoft.AspNetCore": "Warning",
          "Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware": "Information"
        }
      }
    }
    

    or, alternatively, you need to do the same when setting up the logging system in the Program class:

    builder.Services.AddLogging(lb => {
      lb.AddSeq();
    + lb.AddFilter("Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware", LogLevel.Information);
    });
    

    We then have all our pieces in place: let’s execute the application!

    First, you can spin up the API; you should be able to see the Swagger page:

    Swagger page for our application&rsquo;s API

    From here, you can call the GET endpoint:

    Http response of the API call, as seen on Swagger

    You should now able to see all the logs in Seq:

    Logs list in Seq

    As you can see from the screenshot above, I have a log entry for the request and one for the response. Also, of course, I have the custom message I added manually in the C# method.

    Understanding HTTP Request logs

    Let’s focus on the data logged for the HTTP request.

    If we open the log related to the HTTP request, we can see all these values:

    Details of the HTTP Request

    Among these details, we can see properties such as:

    • the host name (localhost:7164)
    • the method (GET)
    • the path (/books/4)

    and much more.

    You can see all the properties as standalone items, but you can also have a grouped view of all the properties by accessing the HttpLog element:

    Details of the HTTP Log element

    Notice that for some elements we do not have access to the actual value, as the value is set to [Redacted]. This is a default configuration that prevents logging too many things (and undisclosing some values) as well as writing too much content on the log sink (the more you write, the less performant the queries become – and you also pay more!).

    Among other redacted values, you can see that even the Cookie value is not directly available – for the same reasons explained before.

    Understanding HTTP Response logs

    Of course, we can see some interesting data in the Response log:

    Details of the HTTP Response

    Here, among some other properties such as the Host Name, we can see the Status Code and the Trace Id (which, as you may notice, is the same as the one in te Request).

    As you can see, the log item does not contain the body of the response.

    Also, just as it happens with the Request, we do not have access to the list of HTTP Headers.

    How to save space, storage, and money by combining log entries

    For every HTTP operation, we end up with 2 log entries: one for the Request and one for the Response.

    However, it would be more practical to have both request and response info stored in the same log item so we can understand more easily what is happening.

    Lucky for us, this functionality is already in place. We just need to set the CombineLogs property to true when we add the HttpLogging functionality:

    builder.Services.AddHttpLogging(lb =>
    {
    +  lb.CombineLogs = true;
    }
    );
    

    Then, we are able to see the data for both the request and the related response in the same log element.

    Request and Response combined logs

    The downsides of using HTTP Logging

    Even though everything looks nice and pretty, adding HTTP Logging has some serious consequences.

    First of all, remember that you are doing some more operations for every incoming HTTP request. Just processing and storing the log messages can bring to an application performance downgrade – you are using parts of the processing resources to interpret the HTTP context, create the correct log entry, and store it.

    Depending on how your APIs are structured, you may need to strip out sensitive data: HTTP Logs, by default, log almost everything (except for the parts stored as Redacted). Since you don’t want to store as plain text the content of the requests, you may need to create custom logic to redact parts of the request and response you want to hide: you may need to implement a custom IHttpLoggingInterceptor.

    Finally, consider that logging occupies storage, and storage has a cost. The more you log, the higher the cost. You should define proper strategies to avoid excessive storage costs while keeping valuable logs.

    Further readings

    There is a lot more, as always. In this article, I focused on the most essential parts, but the road to having proper HTTP Logs is still long.

    You may want to start from the official documentation, of course!

    🔗 HTTP logging in ASP.NET Core | Microsoft Docs

    This article first appeared on Code4IT 🐧

    All the logs produced for this article were stored on Seq. You can find more info about installing and integrating Seq in ASP.NET Core in this article:

    🔗 Easy logging management with Seq and ILogger in ASP.NET | Code4IT

    Wrapping up

    HTTP Logging can be a good tool for understanding the application behaviour and detecting anomalies. However, as you can see, there are some important downsides that need to be considered.

    I hope you enjoyed this article! Let’s keep in touch on LinkedIn or Twitter! 🤜🤛

    Happy coding!

    🐧





    Source link