Extending MySQL using the Component Infrastructure – part 8: linking a third party library

This post is the eight one of a series of articles on extending MySQL with the Component Infrastructure, the list above will be updated as new articles are published:

As explained in post 1, our component will use libclamav. That’s why we need to link it to our component.

The very first thing, we will include clamav.h in our scan.h (line 34):

/* Copyright (c) 2017, 2022, Oracle and/or its affiliates. All rights reserved.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License, version 2.0,
as published by the Free Software Foundation.
This program is also distributed with certain software (including
but not limited to OpenSSL) that is licensed under separate terms,
as designated in a particular file or component or in included license
documentation. The authors of MySQL hereby grant you an additional
permission to link the program and your derivative works with the
separately licensed software that they have included with MySQL.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License, version 2.0, for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
#define LOG_COMPONENT_TAG "viruscan"
#include <mysql/components/component_implementation.h>
#include <mysql/components/services/log_builtins.h> /* LogComponentErr */
#include <mysqld_error.h> /* Errors */
#include <mysql/components/services/dynamic_privilege.h>
#include <mysql/components/services/udf_metadata.h>
#include <mysql/components/services/udf_registration.h>
#include <mysql/components/services/security_context.h>
#include <mysql/components/services/mysql_current_thread_reader.h>
#include <mysql/components/services/mysql_runtime_error_service.h>
#include <list>
#include <string>
#include <clamav.h>
extern REQUIRES_SERVICE_PLACEHOLDER(log_builtins);
extern REQUIRES_SERVICE_PLACEHOLDER(log_builtins_string);
extern REQUIRES_SERVICE_PLACEHOLDER(dynamic_privilege_register);
extern REQUIRES_SERVICE_PLACEHOLDER(udf_registration);
extern REQUIRES_SERVICE_PLACEHOLDER(mysql_udf_metadata);
extern REQUIRES_SERVICE_PLACEHOLDER(mysql_thd_security_context);
extern REQUIRES_SERVICE_PLACEHOLDER(global_grants_check);
extern REQUIRES_SERVICE_PLACEHOLDER(mysql_current_thread_reader);
extern REQUIRES_SERVICE_PLACEHOLDER(mysql_runtime_error);
extern SERVICE_TYPE(log_builtins) * log_bi;
extern SERVICE_TYPE(log_builtins_string) * log_bs;
view raw scan.h hosted with ❤ by GitHub

No we also need to use clamav in our code. I added required structures and variables from lines 46 to 63.

A function called to load the ClamAV virus engine, reload_engine() on line 106. and on line 147, a function that scans the data sent as parameter:

/* Copyright (c) 2017, 2022, Oracle and/or its affiliates. All rights reserved.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License, version 2.0,
as published by the Free Software Foundation.
This program is also distributed with certain software (including
but not limited to OpenSSL) that is licensed under separate terms,
as designated in a particular file or component or in included license
documentation. The authors of MySQL hereby grant you an additional
permission to link the program and your derivative works with the
separately licensed software that they have included with MySQL.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License, version 2.0, for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
#define LOG_COMPONENT_TAG "viruscan"
#define NO_SIGNATURE_CHANGE 0
#define SIGNATURE_CHANGE 1
#include <components/viruscan/scan.h>
REQUIRES_SERVICE_PLACEHOLDER(log_builtins);
REQUIRES_SERVICE_PLACEHOLDER(log_builtins_string);
REQUIRES_SERVICE_PLACEHOLDER(dynamic_privilege_register);
REQUIRES_SERVICE_PLACEHOLDER(udf_registration);
REQUIRES_SERVICE_PLACEHOLDER(mysql_udf_metadata);
REQUIRES_SERVICE_PLACEHOLDER(mysql_thd_security_context);
REQUIRES_SERVICE_PLACEHOLDER(global_grants_check);
REQUIRES_SERVICE_PLACEHOLDER(mysql_current_thread_reader);
REQUIRES_SERVICE_PLACEHOLDER(mysql_runtime_error);
SERVICE_TYPE(log_builtins) * log_bi;
SERVICE_TYPE(log_builtins_string) * log_bs;
static const char *SCAN_PRIVILEGE_NAME = "VIRUS_SCAN";
struct scan_result scan_data(const char *data, size_t data_size);
/*
* Holds the data of a virus scan
*/
struct scan_result
{
int return_code;
const char *virus_name;
long unsigned int scanned;
};
/*
* Global variable to access the ClamAV engine
*/
struct cl_engine *engine = NULL;
char *signatureDir;
struct cl_stat signatureStat;
class udf_list {
typedef std::list<std::string> udf_list_t;
public:
~udf_list() { unregister(); }
bool add_scalar(const char *func_name, enum Item_result return_type,
Udf_func_any func, Udf_func_init init_func = NULL,
Udf_func_deinit deinit_func = NULL) {
if (!mysql_service_udf_registration->udf_register(
func_name, return_type, func, init_func, deinit_func)) {
set.push_back(func_name);
return false;
}
return true;
}
bool unregister() {
udf_list_t delete_set;
/* try to unregister all of the udfs */
for (auto udf : set) {
int was_present = 0;
if (!mysql_service_udf_registration->udf_unregister(udf.c_str(),
&was_present) ||
!was_present)
delete_set.push_back(udf);
}
/* remove the unregistered ones from the list */
for (auto udf : delete_set) set.remove(udf);
/* success: empty set */
if (set.empty()) return false;
/* failure: entries still in the set */
return true;
}
private:
udf_list_t set;
} * list;
unsigned int reload_engine()
{
unsigned int signatureNum = 0;
int rv;
if (engine != NULL)
{
cl_engine_free(engine);
}
engine = cl_engine_new();
memset(&signatureStat, 0, sizeof(struct cl_stat));
signatureDir = const_cast<char*>(cl_retdbdir());
cl_statinidir(signatureDir, &signatureStat);
/*
* Load the signatures from signatureDir, we use only the default dir
*/
rv = cl_load(signatureDir, engine, &signatureNum, CL_DB_STDOPT);
char buf[1024];
if (CL_SUCCESS != rv)
{
sprintf(buf, "failure loading clamav databases: %s", cl_strerror(rv));
LogComponentErr(ERROR_LEVEL, ER_LOG_PRINTF_MSG, buf);
}
rv = cl_engine_compile(engine);
if (CL_SUCCESS != rv)
{
sprintf(buf, "cannot create clamav engine: %s", cl_strerror(rv));
LogComponentErr(ERROR_LEVEL, ER_LOG_PRINTF_MSG, buf);
cl_engine_free(engine);
}
sprintf(buf, "clamav engine loaded with signatureNum %d from %s", signatureNum, signatureDir);
LogComponentErr(INFORMATION_LEVEL, ER_LOG_PRINTF_MSG, buf);
return signatureNum;
}
namespace udf_impl {
struct scan_result scan_data(const char *data, size_t data_size)
{
struct scan_result result = {0, "", 0};
cl_fmap_t *map;
map = cl_fmap_open_memory(data, data_size);
/* scan file descriptor */
static struct cl_scan_options cl_scan_options;
memset(&cl_scan_options, 0, sizeof(struct cl_scan_options));
cl_scan_options.parse |= ~0; /* enable all parsers */
cl_scan_options.general |= CL_SCAN_GENERAL_ALLMATCHES;
result.return_code = cl_scanmap_callback(map,
NULL,
&result.virus_name,
&result.scanned,
engine,
&cl_scan_options,
NULL);
cl_fmap_close(map);
return result;
}
bool have_virus_scan_privilege(void *opaque_thd) {
// get the security context of the thread
Security_context_handle ctx = nullptr;
if (mysql_service_mysql_thd_security_context->get(opaque_thd, &ctx) || !ctx) {
LogComponentErr(ERROR_LEVEL, ER_LOG_PRINTF_MSG,
"problem trying to get security context");
return false;
}
if (mysql_service_global_grants_check->has_global_grant(
ctx, SCAN_PRIVILEGE_NAME, strlen(SCAN_PRIVILEGE_NAME)))
return true;
return false;
}
const char *udf_init = "udf_init", *my_udf = "my_udf",
*my_udf_clear = "my_clear", *my_udf_add = "my_udf_add";
static bool viruscan_udf_init(UDF_INIT *initid, UDF_ARGS *, char *) {
const char* name = "utf8mb4";
char *value = const_cast<char*>(name);
initid->ptr = const_cast<char *>(udf_init);
if (mysql_service_mysql_udf_metadata->result_set(
initid, "charset",
const_cast<char *>(value))) {
LogComponentErr(ERROR_LEVEL, ER_LOG_PRINTF_MSG, "failed to set result charset");
return false;
}
return 0;
}
static void viruscan_udf_deinit(__attribute__((unused)) UDF_INIT *initid) {
assert(initid->ptr == udf_init || initid->ptr == my_udf);
}
const char *viruscan_udf(UDF_INIT *, UDF_ARGS *args, char *outp,
unsigned long *length, char *is_null, char *error) {
MYSQL_THD thd;
mysql_service_mysql_current_thread_reader->get(&thd);
struct scan_result result;
char buf[1024];
if(!have_virus_scan_privilege(thd)) {
mysql_error_service_printf(
ER_SPECIFIC_ACCESS_DENIED_ERROR, 0,
SCAN_PRIVILEGE_NAME);
*error = 1;
*is_null = 1;
return 0;
}
result = scan_data(args->args[0], args->lengths[0]);
if (result.return_code == 0) {
strcpy(outp, "clean: no virus found");
} else {
strcpy(outp, result.virus_name);
sprintf(buf, "Virus found: %s !!", result.virus_name);
LogComponentErr(ERROR_LEVEL, ER_LOG_PRINTF_MSG, buf);
}
*length = strlen(outp);
return const_cast<char *>(outp);
}
} /* namespace udf_impl */
static mysql_service_status_t viruscan_service_init() {
mysql_service_status_t result = 0;
log_bi = mysql_service_log_builtins;
log_bs = mysql_service_log_builtins_string;
LogComponentErr(INFORMATION_LEVEL, ER_LOG_PRINTF_MSG, "initializing…");
int rv;
rv = cl_init(CL_INIT_DEFAULT);
char buf[1024];
if (CL_SUCCESS != rv) {
sprintf(buf, "can't initialize libclamav: %s", cl_strerror(rv));
LogComponentErr(ERROR_LEVEL, ER_LOG_PRINTF_MSG, buf);
} else {
// Print the version of ClamAV engine
sprintf(buf, "ClamAV %s intialized", cl_retver());
LogComponentErr(INFORMATION_LEVEL, ER_LOG_PRINTF_MSG, buf);
}
struct cl_engine *cl_engine_new(void);
reload_engine();
// Registration of the privilege
if (mysql_service_dynamic_privilege_register->register_privilege(SCAN_PRIVILEGE_NAME, strlen(SCAN_PRIVILEGE_NAME))) {
LogComponentErr(ERROR_LEVEL, ER_LOG_PRINTF_MSG,
"could not register privilege 'VIRUS_SCAN'.");
result = 1;
} else {
LogComponentErr(INFORMATION_LEVEL, ER_LOG_PRINTF_MSG,
"new privilege 'VIRUS_SCAN' has been registered successfully.");
}
list = new udf_list();
if (list->add_scalar("virus_scan", Item_result::STRING_RESULT,
(Udf_func_any)udf_impl::viruscan_udf,
udf_impl::viruscan_udf_init,
udf_impl::viruscan_udf_deinit)) {
delete list;
return 1; /* failure: one of the UDF registrations failed */
}
return result;
}
static mysql_service_status_t viruscan_service_deinit() {
mysql_service_status_t result = 0;
cl_engine_free(engine);
if (mysql_service_dynamic_privilege_register->unregister_privilege(SCAN_PRIVILEGE_NAME, strlen(SCAN_PRIVILEGE_NAME))) {
LogComponentErr(ERROR_LEVEL, ER_LOG_PRINTF_MSG,
"could not unregister privilege 'VIRUS_SCAN'.");
result = 1;
} else {
LogComponentErr(INFORMATION_LEVEL, ER_LOG_PRINTF_MSG,
"privilege 'VIRUS_SCAN' has been unregistered successfully.");
}
if (list->unregister()) return 1; /* failure: some UDFs still in use */
delete list;
LogComponentErr(INFORMATION_LEVEL, ER_LOG_PRINTF_MSG, "uninstalled.");
return result;
}
BEGIN_COMPONENT_PROVIDES(viruscan_service)
END_COMPONENT_PROVIDES();
BEGIN_COMPONENT_REQUIRES(viruscan_service)
REQUIRES_SERVICE(log_builtins),
REQUIRES_SERVICE(log_builtins_string),
REQUIRES_SERVICE(dynamic_privilege_register),
REQUIRES_SERVICE(mysql_udf_metadata),
REQUIRES_SERVICE(udf_registration),
REQUIRES_SERVICE(mysql_thd_security_context),
REQUIRES_SERVICE(global_grants_check),
REQUIRES_SERVICE(mysql_current_thread_reader),
REQUIRES_SERVICE(mysql_runtime_error),
END_COMPONENT_REQUIRES();
/* A list of metadata to describe the Component. */
BEGIN_COMPONENT_METADATA(viruscan_service)
METADATA("mysql.author", "Oracle Corporation"),
METADATA("mysql.license", "GPL"), METADATA("mysql.dev", "lefred"),
END_COMPONENT_METADATA();
/* Declaration of the Component. */
DECLARE_COMPONENT(viruscan_service,
"mysql:viruscan_service")
viruscan_service_init,
viruscan_service_deinit END_DECLARE_COMPONENT();
/* Defines list of Components contained in this library. Note that for now
we assume that library will have exactly one Component. */
DECLARE_LIBRARY_COMPONENTS &COMPONENT_REF(viruscan_service)
END_DECLARE_LIBRARY_COMPONENTS
view raw scan.cc hosted with ❤ by GitHub

The first function that load the engine (reload_engine()) is called when our component is loaded in viruscan_service_init() on line 261.

The second function, scan_data(), is called when we use our function viruscan_udf(), on line 223 and we add in the output buffer (outp) if we found virus or not.

In case we find a virus, we also add a message in error log (line 229).

We also fixed the previous warnings we had during compilation (line 218 and 219).

On line 289, we free the ClamAV engine when the component is uninstalled.

We can compile and test the code:

Oups… we got a undefined_symbol error !

This is because we also nee to tell the compiler to link clamav libraries. We add it to CmakeLists.txt as below:

DISABLE_MISSING_PROFILE_WARNING()

MYSQL_ADD_COMPONENT(viruscan
  scan.cc 
  MODULE_ONLY
  TEST_ONLY
  LINK_LIBRARIES clamav
  )

After the change we can run cmake and make again.

Let’s test it, you will notice that installing the component takes time (+/- 15 sec on my system)… this is because it load the virus database:

The ClamAV engine version is returned on line 256 and displayed on line 257 to error log. This is another example with a more recent version:

Now let’s test the scan of data:

On the next article we will add a second function to our component to be able to reload the Clamav Virus Engine in case it gets updated (you usually refresh your virus database more often than you restart your MySQL Server).

Keep coding and as usual, enjoy MySQL !

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

One comment

Leave a Reply

Your email address will not be published. Required fields are marked *

As MySQL Community Manager, I am an employee of Oracle and the views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

You can find articles I wrote on Oracle’s blog.

Continue