JavaScript Rendering

Use the built-in Chrome DevTools Protocol to render JavaScript-heavy pages without external services.

config.toml

name = "JS Rendering Example"
url = "https://news.ycombinator.com"
selector = ".athing"
fields = [
  "title:.titleline > a",
  "link:.titleline > a@href"
]
interval = 300

hooks.lua

if not browser then
    browser = cdp.launch({
        headless = true,
        keep_alive = true,
    })
end

function override_fetch(request)
    local page = browser:attach()
    defer(function() page:close() end)

    local ok, err = page:open(request.url)
    if not ok then
        return { error = "Failed to open: " .. tostring(err) }
    end

    page:wait_for_selector(".js-rendered-item", 10000)
    page:screenshot("last_render.png")

    return {
        status = 200,
        body = page:content(),
        url = request.url
    }
end

Key Concepts

• cdp.launch() starts a persistent Chromium browser
• defer() ensures the page closes even if the hook errors
• wait_for_selector() blocks until the element appears or timeout
• Global browser persists across runs for faster subsequent fetches