WageDeterminationController.cs Documentation 📝🚀

File: IrbyImaging/UI/Controllers/WageDeterminationController.cs


Table of Contents

  1. Overview & Purpose
  2. Key Functional Areas
    1. Index Action
    2. GetWages Action
    3. Helper Methods
  3. Dependencies & Technologies
  4. Data Models & View Models
  5. API Endpoints & Routes
  6. Error Handling & Return Types
  7. Third‑Party Integrations
  8. Sample Usage
  9. UML Class Diagram
  10. Conclusion

Overview & Purpose

The WageDeterminationController is part of the IrbyImaging UI layer. Its primary goal is to:

  • 🎯 Fetch Davis‑Bacon Act (DBA) and Service Contract Act (SCA) wage determinations from SAM.gov.
  • 🖥️ Automate form‑filling and scraping using Selenium WebDriver (headless Chrome).
  • 📄 Extract the wage determination section as formatted HTML.
  • 📊 Present the results in an ASP.NET Core MVC view (Wages model).

This approach is used because SAM.gov does not provide a public API for direct wage‑determination queries, so web‑scraping is required.


Key Functional Areas

1. Index Action

public ActionResult Index()
{
    // Prepare dropdown categories
    var category = new List<SelectListItem>
    {
        new SelectListItem { Value = "Davis-Bacon Act (DBA)", Text = "Davis-Bacon Act (DBA)" },
        new SelectListItem { Value = "Service Contract Act (SCA)", Text = "Service Contract Act (SCA)" }
    };
    ViewBag.categories = category;

    // Initialize an empty Wages model for the form
    var wages = new Wages();
    return View(wages);
}
  • Purpose:
    • Initializes the category dropdown (DBA vs. SCA).
    • Returns the Index view with a fresh Wages model for user input.

2. GetWages Action

Signature:
public ActionResult GetWages(Wages wage)

Flow:

  1. Select Base URL

    • Default is Davis‑Bacon search.
    • If wage.Category == "Service Contract Act (SCA)", switch to SCA index.
  2. Initialize Headless Chrome

    var chromeOptions = new ChromeOptions();
    chromeOptions.AddArgument("--headless");
    chromeOptions.AddArgument("--disable-gpu");
    using (IWebDriver driver = new ChromeDriver(chromeOptions))
    {
        // ...
    }
    
  3. Navigate & Handle Initial Popup

    • Go to baseUrl.
    • Wait for sds-dialog-container and click Close.
  4. Fill Filters

    • State: CSS input#wd-state.
    • County: CSS input#wd-county.
    • Construction Type (DBA only): CSS input#dba-construction-type.
    • Each field: scroll into view, click, type, press Enter.
  5. Refresh & Locate Result

    • After filters, retrieve updated driver.Url.
    • Navigate to it again to ensure filters applied.
    • Wait for an <h3> containing "Davis‑Bacon Act WD #:".
    • Extract its <a.usa-link> href and build full URL.
  6. Extract Wage Determination Section

    • Navigate to detail page.
    • Wait for <section id="wage-determination">.
    • Grab outerHTML, run through PrettyPrintHtml(...).
    • Assign to wage.Content.
  7. Return View

    return View(wage);
    

Highlights:

  • Uses Selenium WebDriver in headless mode.
  • Uses WebDriverWait to synchronize.
  • Applies System.Threading.Thread.Sleep(...) as a fallback for timing.
  • Safety: driver.Quit() is called to free resources.

3. Helper Methods

Method Description
SetDropdownValue(...) Sets <select> or <input> via JavaScript, dispatches input & change events.
static string PrettyPrintHtml(...) Uses HtmlAgilityPack to parse & write formatted HTML.
HandlePopupDialog(...) Generic logic to close SAM.gov popup by ID and click OK.
static string PrettyPrintHtml(string html)
{
    var doc = new HtmlDocument();
    doc.LoadHtml(html);
    using (var writer = new System.IO.StringWriter())
    {
        doc.Save(writer);
        return writer.ToString();
    }
}

Dependencies & Technologies

Category Tools / Libraries
Web Framework ASP.NET Core MVC, Microsoft.AspNetCore.Mvc
Configuration & DI Microsoft.Extensions.Configuration
Data Access Entity Framework Core (IrbyContext)
JSON Newtonsoft.Json, Newtonsoft.Json.Linq
WebDriver / Scraping Selenium WebDriver, ChromeDriver, OpenQA.Selenium.*
HTML Parsing HtmlAgilityPack (HtmlAgilityPack)
UI Helpers Microsoft.AspNetCore.Mvc.Rendering.SelectListItem
(Imported but unused) Azure.Storage.Blobs, DocumentFormat.OpenXml.*

Data Models & View Models

Wages (ViewModel)

namespace IrbyImaging.UI.Models
{
    public class Wages
    {
        public string State { get; set; }
        public string County { get; set; }
        public string Category { get; set; }
        public string ConstructionType { get; set; }
        public string Content { get; set; }    // HTML of wage determination section
    }
}
Property Type Description
State string Two‑letter state code (e.g., "CA")
County string County name (e.g., "Los Angeles")
Category string "Davis-Bacon Act (DBA)" or "Service Contract Act (SCA)"
ConstructionType string (DBA only) e.g., "Building", "Highway"
Content string Scraped HTML content of the determination

API Endpoints & Routes

Route HTTP Verb Description
/WageDetermination/Index GET Renders input form for state, county, and category.
/WageDetermination/GetWages POST Scrapes SAM.gov and returns Wages.Content view.

NOTE: The [HttpPost] attribute is commented out – ensure you enable it for production use.


Error Handling & Return Types

  • Return Type: All actions return ActionResult.
  • Pop‑up/Dropdown Helpers: Wrapped in try/catch, logging exceptions to console:
    catch (Exception ex) 
    {
        Console.WriteLine($"Error setting dropdown value for {dropdownId}: {ex.Message}");
    }
    
  • Failure Modes:
    • If Selenium cannot locate elements, the user sees no data or a server error.
    • Consider adding user‑friendly error messages or retry logic.

Third‑Party Integrations

  • ✅ Selenium WebDriver (Chrome) – automates browser interactions.
  • ✅ HtmlAgilityPack – parses and formats scraped HTML.
  • ℹ️ SAM.gov – source of Davis‑Bacon & SCA wage determinations.
  • ⚙️ Entity Framework Core – IrbyContext for possible logging or caching (not shown here).

Sample Usage

1. Razor View (Index.cshtml)

@model IrbyImaging.UI.Models.Wages

<form asp-action="GetWages" method="post">
  <div>
    <label>Category</label>
    <select asp-for="Category" asp-items="ViewBag.categories"></select>
  </div>
  <div>
    <label>State</label>
    <input asp-for="State" placeholder="e.g. CA" />
  </div>
  <div>
    <label>County</label>
    <input asp-for="County" placeholder="e.g. Los Angeles" />
  </div>
  <div>
    <label>Construction Type (DBA only)</label>
    <input asp-for="ConstructionType" />
  </div>
  <button type="submit">🔎 Get Wage Determination</button>
</form>

@if (Model != null && !string.IsNullOrEmpty(Model.Content))
{
  <section>
    <h2>Wage Determination:</h2>
    @Html.Raw(Model.Content)
  </section>
}

2. Controller Invocation (Unit‑Test Style)

// Arrange
var context = /* mock or in‑memory IrbyContext */;
var config  = /* IConfiguration */;
var controller = new WageDeterminationController(context, config);

// Act
var wageRequest = new Wages 
{ 
  State = "CA", 
  County = "Los Angeles", 
  Category = "Davis-Bacon Act (DBA)", 
  ConstructionType = "Building" 
};

var result = controller.GetWages(wageRequest) as ViewResult;

// Assert
var model = result.Model as Wages;
Assert.NotNull(model);
Assert.Contains("<section", model.Content);

UML Class Diagram

classDiagram
    class WageDeterminationController {
        - IrbyContext _context
        - IConfiguration _configuration
        - SamGov _samGovService
        - string baseUrl
        - string baseWageUrl
        + Index() : ActionResult
        + GetWages(Wages) : ActionResult
        - SetDropdownValue(IWebDriver,WebDriverWait,string,string) : void
        - PrettyPrintHtml(string) : string
        - HandlePopupDialog(IWebDriver,WebDriverWait) : void
    }

    class Wages {
        + string State
        + string County
        + string Category
        + string ConstructionType
        + string Content
    }

    WageDeterminationController --> Wages : uses

Conclusion

The WageDeterminationController provides a robust, automated way to retrieve federal wage determinations (DBA & SCA) from SAM.gov by leveraging:

  • 🔄 Headless Chrome & Selenium for interactive scraping.
  • 🖋️ HtmlAgilityPack to tidy up the HTML.
  • 🎨 ASP.NET Core MVC to present a user-friendly form and results.

Though effective, be mindful of:

  • Site Changes: SAM.gov markup updates may break selectors.
  • Performance: Spawning ChromeDriver is heavy; consider caching.
  • Ethics & Terms: Ensure scraping complies with SAM.gov usage policies.

Now you’re equipped to integrate, extend, and maintain wage‑determination scraping in your .NET applications! 🚀🖥️


Happy coding! 🎉