I’ve written a service for our application which looks up URLs against the Google Safer Browsing API via the official node.js client.

The issue we’re encountering is that all URLs that flag with Google Safer Browsing via the browser or Google Transparency report return an empty data object via our service, with the exception of Google’s test URL ([http]malware[dot]testing[dot]google[dot]test/testing/malware/*). The HTTP response is ‘200’, which rules out a key misconfiguration.

I’ve found plenty of examples where others have had the same issue, with no resolution. My service code is shown below:

import { Response } from './interfaces/response';
import { google } from 'googleapis';

export class GoogleSafeBrowsingBaseService {
  protected getReport = async (url: string): Promise<Response> => {
    const safebrowsing = google.safebrowsing({ version: 'v4', auth: process.env.GSB_KEY });

    return safebrowsing.threatMatches
      .find({
        requestBody: {
          client: {
            clientId: process.env.GSB_CLIENT,
            clientVersion: process.env.GSB_VER,
          },
          threatInfo: {
            threatTypes: [
              'MALWARE',
              'SOCIAL_ENGINEERING',
              'UNWANTED_SOFTWARE',
              'POTENTIALLY_HARMFUL_APPLICATION',
            ],
            platformTypes: ['ANY_PLATFORM', 'PLATFORM_TYPE_UNSPECIFIED'],
            threatEntryTypes: ['URL'],
            threatEntries: this.getUrlVariants(url).map((variant) => ({
              url: variant,
            })),
          },
        },
      })
      .then(({ data }) =>
        Object.keys(data).length ? <Response>data : <Response>{ matches: [] },
      );
  };

  private getUrlVariants = (url: string): string[] => {
    const parsedUrl = new URL(url.trim());
    parsedUrl.hash = '';

    if (!parsedUrl.pathname.endsWith('/')) {
      parsedUrl.pathname += '/';
    }

    const results: string[] = [
      `http://${parsedUrl.host}/`,
      `https://${parsedUrl.host}/`,
      `http://${parsedUrl.host}${parsedUrl.pathname}`,
      `https://${parsedUrl.host}${parsedUrl.pathname}`,
    ];

    if (parsedUrl.search) {
      results.push(
        `http://${parsedUrl.host}${parsedUrl.pathname}${parsedUrl.search}`,
        `https://${parsedUrl.host}${parsedUrl.pathname}${parsedUrl.search}`,
      );
    }

    return results;
  };
}

As per documentation, inputted URLs are:

• Stripped of fragments, ie #someanchor
• URLs are formatted with a trailing slash
• Whitespace is stripped (although our controller does already do this)

We’ve also ensured:

• The API key we’re using is correctly locked down to our server
• The API key is configured for Google’s Safer Browsing in Google Cloud Console
• The values of GSB_CLIENT, GSB_VER and GSB_KEY do match those of the response object

Any information to resolve this would be gratefully appreciated.

Examples of URLs we’ve tried with our service, (which at time of writing) do flag up as malicious in some way on Google Transparency Report include:

• [http]business[dot]standardcontactverify[dot]com/case/113458020784789 – HTTP 200, empty data property (expected data property populated with object containing the matches)
• [http]malware[dot]testing[dot]google[dot]test/testing/malware/* – Google’s test URL, HTTP 200 with populated data property (response as expected)
• [https]business[dot]verifysecuritypolicy[dot]com/case/113458020784789 – HTTP 200, empty data property (expected data property populated with object containing the matches)