C++: std::function, implicit conversions, ambiguous overloads, and a bit of a mess
-
This is pretty long, please with me.
Since the "minimal working example" to demonstate the problem is a bit complicated, I'll build up from the simplest case and add code and requirements in several steps from there.I'm writing a class where the user has to pass in several functions, and want to make the use of that class as easy as possible for the user. I arrived at a bit of an impasse now, so maybe one of you can help.
Let's start with the basics:
The user has to pass in two function callbacks (with different type) to my class (in reality it's 4, but let's keep it simple). For maximum flexibility, the interface uses the type-erasure ofstd::function
, so the user can pass in any kind of callable, especially closures etc.For now, that's easy enough:
using std::string; // The types of functions template <typename T> using Fun = std::function<T(T, bool)>; using FunI = Fun<int>; using FunS = Fun<string>; // the "library" class to provide some functionality class NeedsFunctions { FunI fi; FunS fs; void run() { fi(0, true); fs("", true); } // dummy calls to the functions public: NeedsFunctions(FunI fi_, FunS fs_) : fi(fi_), fs(fs_) { run(); } }; // example client code int fooI(int x, bool) { return x; } string fooS(string x, bool) { return x; } template <typename T> T foo(T x, bool) { return x; } void test() { // // Functions with 2 parameters // // OK: 2 free functions NeedsFunctions nf1(fooI, fooS); // OK: 2 template instantiations NeedsFunctions nf2(foo<int>, foo<string>); }
As can be seen in the example, the user will often want to pass basically the same function twice, but with different types. So it would be nice if, instead of passing the two parameters
foo<int>, foo<string>
, he could just passfoo
. Unfortunately, as far as I know, a restriction of C++ is that you can't pass function templates in any way, but e.g. class templates work. Thus, the idea is that you can provide a single parameter which can act as a callable for both types. (Obviously, you could write a class with twooperator()
, but then you haven't gained anything. But you can provide a templatedoperator()
or just achieve it with a generic lambda, which does this under the hood)So, add this additional constructor
// Constructor template that constructs both functions from a single argument, // usually a generic callable. template <typename T> NeedsFunctions(const T& f) : fi(f), fs(f) { run(); }
and then client code can deal with these two further calls:
// OK: 1 generic lambda auto bar = [](auto x, bool){ return x; }; NeedsFunctions nf3(bar); // OK: 1 generic lambda, with closure int captured = 0; auto baz = [&captured](auto x, bool){ captured++; return x; }; NeedsFunctions nf4(baz); std::cout << "captured: " << captured << std::endl; // outputs "2"
Full code
using std::string; // The types of functions template <typename T> using Fun = std::function<T(T, bool)>; using FunI = Fun<int>; using FunS = Fun<string>; class NeedsFunctions { FunI fi; FunS fs; void run() { fi(0, true); fs("", true); } // dummy calls to the functions public: // Normal constructor that actually gets what we needs NeedsFunctions(FunI fi_, FunS fs_) : fi(fi_), fs(fs_) { run(); } // Constructor template that constructs both functions from a single argument, // usually a generic callable. template <typename T> NeedsFunctions(const T& f) : fi(f), fs(f) { run(); } }; int fooI(int x, bool) { return x; } string fooS(string x, bool) { return x; } template <typename T> T foo(T x, bool) { return x; } int simpleI(int x, bool) { return x; } string simpleS(string x, bool) { return x; } template <typename T> T simple(T x, bool) { return x; } void test() { // // Functions with 2 parameters // // OK: 2 free functions NeedsFunctions nf1(fooI, fooS); // OK: 2 template instantiations NeedsFunctions nf2(foo<int>, foo<string>); // OK: 1 generic lambda auto bar = [](auto x, bool){ return x; }; NeedsFunctions nf3(bar); // OK: 1 generic lambda, with closure int captured = 0; auto baz = [&captured](auto x, bool){ captured++; return x; }; NeedsFunctions nf4(baz); std::cout << "captured: " << captured << std::endl; // outputs "2" }
This is nice, but now comes a new requirement: The callback functions all have two parameters, but often the second one (
bool
) isn't needed, so for the user it would be more convenient (and maybe less surprising) if he didn't have to declare his callback with an unneccessary parameter.
For this, my idea was to have a wrapper class that takes a "simplified" function with one parameter and automatically creates a two-parameter wrapper that simply discards the unneccesary one.Add the following wrapper
template <typename T> using SimpleFun = std::function<T(T)>; using SimpleI = SimpleFun<int>; using SimpleS = SimpleFun<string>; template <class T> class FunctionWrapper { Fun<T> fun; public: // no-op wrapper, Fun -> Fun FunctionWrapper(const Fun<T>& f) : fun(f) {} // simplifying fun wrapper, SimpleFun -> Fun FunctionWrapper(const SimpleFun<T>& f) : fun([=](T x, bool) { return f(x); }) {} operator Fun<T>() const { return fun; } }; using FunctionWrapperI = FunctionWrapper<int>; using FunctionWrapperS = FunctionWrapper<string>;
change the constructors of the class, and add the following new examples:
int simpleI(int x) { return x; } string simpleS(string x) { return x; } template <typename T> T simple(T x) { return x; } void test() { // ... snip ... // // Simple functions with 1 parameter // // OK: 2 free "simple" functions NeedsFunctions nf5(simpleI, simpleS); // OK: 2 template instantiations NeedsFunctions nf6(simple<int>, simple<string>); // OK: 1 "simplified" generic lambda. auto simpleBar = [](auto x){ return x; }; NeedsFunctions nf7(simpleBar); // OK: 1 "simplified" generic lambda, with closure. int simpleC = 0; auto simpleBaz = [&simpleC](auto x){ simpleC++; return x; }; NeedsFunctions nf8(simpleBaz); std::cout << "simpleC: " << simpleC << std::endl; // outputs "2"
Full code
using std::string; // The types of functions template <typename T> using Fun = std::function<T(T, bool)>; using FunI = Fun<int>; using FunS = Fun<string>; template <typename T> using SimpleFun = std::function<T(T)>; using SimpleI = SimpleFun<int>; using SimpleS = SimpleFun<string>; template <class T> class FunctionWrapper { Fun<T> fun; public: // no-op wrapper, Fun -> Fun FunctionWrapper(const Fun<T>& f) : fun(f) {} // simplifying fun wrapper, SimpleFun -> Fun FunctionWrapper(const SimpleFun<T>& f) : fun([=](T x, bool) { return f(x); }) {} operator Fun<T>() const { return fun; } }; using FunctionWrapperI = FunctionWrapper<int>; using FunctionWrapperS = FunctionWrapper<string>; class NeedsFunctions { FunI fi; FunS fs; void run() { fi(0, true); fs("", true); } // dummy calls to the functions public: // Normal constructor that actually gets what we needs NeedsFunctions(FunI fi_, FunS fs_) : fi(fi_), fs(fs_) { run(); } // Constructor template that constructs both functions from a single argument, // usually a generic callable. template <typename T> NeedsFunctions(const T& f) : fi(FunctionWrapperI(f)), fs(FunctionWrapperS(f)) { run(); } // Constructor that accepts "simplified" functions with only 1 parameter NeedsFunctions(SimpleI fi_, SimpleS fs_) : fi(FunctionWrapperI(fi_)), fs(FunctionWrapperS(fs_)) { run(); } }; int fooI(int x, bool) { return x; } string fooS(string x, bool) { return x; } template <typename T> T foo(T x, bool) { return x; } int simpleI(int x) { return x; } string simpleS(string x) { return x; } template <typename T> T simple(T x) { return x; } void test() { // // Functions with 2 parameters // // OK: 2 free functions NeedsFunctions nf1(fooI, fooS); // OK: 2 template instantiations NeedsFunctions nf2(foo<int>, foo<string>); // OK: 1 generic lambda auto bar = [](auto x, bool){ return x; }; NeedsFunctions nf3(bar); // OK: 1 generic lambda, with closure int captured = 0; auto baz = [&captured](auto x, bool){ captured++; return x; }; NeedsFunctions nf4(baz); std::cout << "captured: " << captured << std::endl; // outputs "2" // // Simple functions with 1 parameter // // OK: 2 free "simple" functions NeedsFunctions nf5(simpleI, simpleS); // OK: 2 template instantiations NeedsFunctions nf6(simple<int>, simple<string>); // OK: 1 "simplified" generic lambda. Doesn't match any ctor auto simpleBar = [](auto x){ return x; }; NeedsFunctions nf7(simpleBar); // OK: 1 "simplified" generic lambda, with closure. Doesn't match any ctor int simpleC = 0; auto simpleBaz = [&simpleC](auto x){ simpleC++; return x; }; NeedsFunctions nf8(simpleBaz); std::cout << "simpleC: " << simpleC << std::endl; // outputs "2" }
This all works so far, but now comes the point where I get stuck.
What we've got now is the ability to pass either
- 2 functions, or
- one generic callable that gets used to create two function, or
- 2 "simplified" functions, or
- one generic callable for the "simplified" version
But once you start overcomplicating the library to make the client code simpler, you might as well go for the full deal. ;)
Now I would like the user to be able to pass any combination of "full" and "simple" functions.
// // Mix and match // // Error: no matching ctor NeedsFunctions nf9(fooI, simpleS);
As we already have constructors for "full, full" and "simple, simple", it would seem that we only have to add two more combinations. But, as said above, in my real use case I have 4 functions (actually, 2 pairs of functions, for the generic use case), and I would really prefer to avoid the combinatoric explosion of different constructors.
So my idea here was: instead of the constructor taking either a
Fun
or aSimpleFun
, and using theFunctionWrapper
to convert the latter to the former, just use a single constructor that takes aFunctionWrapper
to begin with. Clever, isn't it? At least so I thought:// Normal constructor that actually gets what we needs NeedsFunctions(FunctionWrapperI fi_, FunctionWrapperS fs_) : fi(fi_), fs(fs_) { run(); } // Constructor template that constructs both functions from a single argument, // usually a generic callable. template <typename T> NeedsFunctions(const T& f) : fi(FunctionWrapperI(f)), fs(FunctionWrapperS(f)) { run(); }
However, now our very first example
nf1
fails witherror: no matching function for call to ‘NeedsFunctions::NeedsFunctions(int (&)(int, bool), std::string (&)(std::string, bool))’
. This is because the free function implicitly converts toFun
(aka std::function), andFun
implicitly converts toFunctionWrapper
, but you can't chain two implicit conversions. So now, for the code to compile, you'd have to write explicitlyNeedsFunctions nf1(FunctionWrapperI(fooI), FunctionWrapperS(fooS));
That's pretty ugly and a step backwards for usability. I'd rather abandon that.
But I had one more idea: Skip thestd::function
middle man and have theFunctionWrapper
accept free functions directly. Add the following code:using PureFun = T(T, bool); // directly accept free functions FunctionWrapper(PureFun f) : fun(f) {}
and example 1 now compiles without the additional explicit constructor calls, as before.
Unfortunately, now I've introduced an ambiguous call for the third example:
error: call of overloaded ‘FunctionWrapper(const test()::<lambda(auto:12, bool)>&)’ is ambiguous
Now I'm out of ideas how to achieve the implicit conversion without adding ambiguity to the constructors.
Full code
using std::string; template <typename T> using Fun = std::function<T(T, bool)>; using FunI = Fun<int>; using FunS = Fun<string>; template <typename T> using SimpleFun = std::function<T(T)>; using SimpleI = SimpleFun<int>; using SimpleS = SimpleFun<string>; template <class T> class FunctionWrapper { Fun<T> fun; using PureFun = T(T, bool); public: // no-op wrapper, Fun -> Fun FunctionWrapper(const Fun<T>& f) : fun(f) {} // simplifying fun wrapper, SimpleFun -> Fun FunctionWrapper(const SimpleFun<T>& f) : fun([=](T x, bool) { return f(x); }) {} // directly accept free functions FunctionWrapper(PureFun f) : fun(f) {} operator Fun<T>() const { return fun; } }; using FunctionWrapperI = FunctionWrapper<int>; using FunctionWrapperS = FunctionWrapper<string>; class NeedsFunctions { FunI fi; FunS fs; void run() { fi(0, true); fs("", true); } // dummy calls to the functions public: // Normal constructor that actually gets what we needs NeedsFunctions(FunctionWrapperI fi_, FunctionWrapperS fs_) : fi(fi_), fs(fs_) { run(); } // Constructor template that constructs both functions from a single argument, // usually a generic callable. template <typename T> NeedsFunctions(const T& f) : fi(FunctionWrapperI(f)), fs(FunctionWrapperS(f)) { run(); } }; int fooI(int x, bool) { return x; } string fooS(string x, bool) { return x; } template <typename T> T foo(T x, bool) { return x; } int simpleI(int x) { return x; } string simpleS(string x) { return x; } template <typename T> T simple(T x) { return x; } void test() { // // Functions with 2 parameters // // OK: 2 free functions NeedsFunctions nf1(fooI, fooS); // OK: 2 template instantiations NeedsFunctions nf2(foo<int>, foo<string>); // OK: 1 generic lambda auto bar = [](auto x, bool){ return x; }; NeedsFunctions nf3(bar); // OK: 1 generic lambda, with closure int captured = 0; auto baz = [&captured](auto x, bool){ captured++; return x; }; NeedsFunctions nf4(baz); std::cout << "captured: " << captured << std::endl; // outputs "2" // // Simple functions with 1 parameter // // OK: 2 free "simple" functions NeedsFunctions nf5(simpleI, simpleS); // OK: 2 template instantiations NeedsFunctions nf6(simple<int>, simple<string>); // OK: 1 "simplified" generic lambda. Doesn't match any ctor auto simpleBar = [](auto x){ return x; }; NeedsFunctions nf7(simpleBar); // OK: 1 "simplified" generic lambda, with closure. Doesn't match any ctor int simpleC = 0; auto simpleBaz = [&simpleC](auto x){ simpleC++; return x; }; NeedsFunctions nf8(simpleBaz); std::cout << "simpleC: " << simpleC << std::endl; // outputs "2" // // Mix and match // // Error: no matching ctor NeedsFunctions nf9(fooI, simpleS); }
EDIT: Creating an "evolution" of this code from one source and keeping the parts consistent was kind of hard, so of course I messed up. Obviously, the "simple" functions (
simpleI
,simpleS
) all should have one parameter only, not just in some snippets.
-
(attempt two)
The following works in C++14 (for realsis now); if you have a C++17 capable compiler with support for
std::is_invocable<>
, you should probably prefer that over theis_callable<>
here.Anyway,
is_callable
detects if an instance of a certain type is callable with arguments of some given type. So, for example,is_callable< int(int), int >
would be true, whereasis_callable< int(int), std::string >
wouldn't.I then use SFINAE via
std::enable_if_t
to choose between two possible constructors forFunctionWrapper
, one that accepts things callable with just aT
argument, and one that accepts callable things withT, bool
.New
FunctionWrapper<>
:template <class T> class FunctionWrapper { Fun<T> fun; public: template< class tCallable > FunctionWrapper( tCallable&& aCallable, std::enable_if_t< is_callable<tCallable,T>::value, int > = 0 ) : fun( [f = std::forward<tCallable>(aCallable)](T x, bool) { return f(x); } ) {} template< class tCallable > FunctionWrapper( tCallable&& aCallable, std::enable_if_t< is_callable<tCallable,T,bool>::value, int > = 0 ) : fun( std::forward<tCallable>(aCallable) ) {} operator Fun<T>() const { return fun; } };
is_callable
implementation:namespace detail { template< typename... > struct Identity {}; // See https://en.cppreference.com/w/cpp/types/void_t template<typename... Ts> struct make_void { typedef void type;}; template<typename... Ts> using void_t = typename make_void<Ts...>::type; template< class, class, class = void_t<> > struct is_callable0_ : std::false_type {}; template< class tCallable, typename... tArgs > struct is_callable0_< tCallable, Identity<tArgs...>, void_t< decltype(std::declval<tCallable>()(std::declval<tArgs>()...)) > > : std::true_type {}; } template< class tCallable, typename... tArgs > struct is_callable : detail::is_callable0_< tCallable, detail::Identity<tArgs...> > {};
Edited: I messed up before and copy-pasted the wrong side of an #ifdef. :-/
-
And a small improvement if you don't already do that in the real code:
- in FunctionWrapper:
operator Fun<T>&&() && { return std::move(fun); }
- in NeedsFunctions:
NeedsFunctions(FunctionWrapperI fi_, FunctionWrapperS fs_) : fi(std::move(fi_)), fs(std::move(fs_)) ...
should avoid copying
std::function
unnecessarily.
- in FunctionWrapper:
-
@cvi Cool, thanks!!
I've not tried it yet (I'm done with work for today ;) ), but a quick question for my understanding:
It solves the two-implicit-conversions problem by directly accepting something more general than astd::function
, but does so without creating ambiguity (like my solution did for things that are neither exactly free functions norstd::function
)?
-
@topspin Yeah, kinda.
The two overloads of the
FunctionWrapper
constructor are -to begin with- ambiguous, since they both accept atCallable
, which could be anything. But theenable_if_t
removes overloads via SFINAE if the conditions don't apply (in short,enable_if_t<b,T> = typename enable_if<b,T>::type
; but ifb
is false, thenenable_if<>
doesn't define the nestedtype
field, and asking for it becomes an error that triggers SFINAE if used in the right context ... more or less).The condition that I use is that
tCallable
is callable with just aT
andT, bool
, respectively. That does mean that something likestruct X { int operator() (int); int operator() (int, bool); }; FunctionWrapper<int> fwi( X{} );
again becomes ambiguous, since X is callable either way. But now you can fix that by augmenting one of the conditions to explicitly excluding the other options. Like, changing:
template< class tCallable > FunctionWrapper( tCallable&& aCallable, std::enable_if_t< is_callable<tCallable,T>::value, int > = 0 ) : fun( [f = std::forward<tCallable>(aCallable)](T x, bool) { return f(x); } ) {}
to
template< class tCallable > FunctionWrapper( tCallable&& aCallable, std::enable_if_t< is_callable<tCallable,T>::value && !is_callable<tCallable,T,bool>::value, int > = 0 ) : fun( [f = std::forward<tCallable>(aCallable)](T x, bool) { return f(x); } ) {}
(note- unlike the previous code samples, I've not tested the stuff here. The idea should work, I think.)
tl;dr: You can use
enable_if_t
to manually eliminate ambiguities. At least if you can identify the ambiguity and formulate a condition to disambiguate it. (IME it's often easier to start with something very general and narrow it down in different ways, rather than with two distinct things that only overlap occasionally. Thus the it-could-be-anythingtCallable
as a starting point.)
-
@cvi Yeah, I know how SFINAE /
enable_if
work, just wanted to make sure that that's the idea behind your solution.I actually tried to come up with something like that myself, but I seem to remember than when I started with just a single, completely general constructor
template <typename T> FunctionWrapper(T f)
the free function wouldn't bind toT
, so I assumed you have to go down the function syntax road ofT(T, bool)
. Obviously though, you did exactly that, so I must've gotten something wrong that it didn't bind. Maybe the rvalue qualifier?
I'll try again tomorrow.Thanks a lot!
-
@cvi Another question, while I'm at it:
If you pass in atCallable
argumentf
that is already aFun<T>
(i.e. the type it should convert to, as in the "no-op" constructor), will the compiler be literal and create functiong
with capturedf
and call that in a chain ofstd::function
s, or is the optimizer smart enough to see through that and collapse the two?
-
@topspin said in C++: std::function, implicit conversions, ambiguous overloads, and a bit of a mess:
Maybe the rvalue qualifier?
Could be. I've run into similar issues before, but don't remember the details just now. The argument is a forwarding/universal reference and not a rvalue reference in this context. With that, the type for free functions end up being
T (&)(T, bool)
... maybe you can't have aT(T,bool)
value, but it has to either be a reference or a pointer? That would kinda make sense.If you pass in a
tCallable
argumentf
that is already aFun<T>
(i.e. the type it should convert to, as in the "no-op" constructor), will the compiler be literal and create functiong
with capturedf
and call that in a chain ofstd::function
s, or is the optimizer smart enough to see through that and collapse the two?No, that should invoke either the copy or the move constructor of
Fun<T>
, so it will not end up chaining multiplestd::function
s. (The code uses perfect forwarding, so it should prefer the move constructor whenever it's possible to do so.)
-
@cvi said in C++: std::function, implicit conversions, ambiguous overloads, and a bit of a mess:
The code uses perfect forwarding, so it should prefer the move constructor whenever it's possible to do so.
It uses perfect forwarding to move into the capture, but (unless the optimizer can prevent that) still generates a function from a lambda which captures the passed in original function, as far as I can tell.
To be clear about my question: I'm not worried about the performance of the construction of the FunctionWrapper, either copy or move, doesn't matter. I'm just wondering if calling the result (after you retrieve it with
operator Fun<T>()
) will be as fast as if it hadn't gone through the wrapper, or if it ends up with an additional indirection.
-
@topspin said in C++: std::function, implicit conversions, ambiguous overloads, and a bit of a mess:
It uses perfect forwarding to move into the capture, but (unless the optimizer can prevent that) still generates a function from a lambda which captures the passed in original function, as far as I can tell.
Oh, yeah, that. Sorry, misunderstood - was thinking about the other constructor.
Yeah, if you pass in a
function<int(int)>
it will capture that (=either copy or move), store it in the lambda, and create an outerfunction<int(int,bool)>
around that. So there's an indirection in that case, but that would have been there with your method as well.If you pass in a
int(int)
free function or function pointer, it will not create an innerfunction<int(int)>
, though. Neither will it do that for a callable object -- it will capture the callable object (copy or move) directly, and not via an innerfunction<int(int)>
.I think original approach would result in an inner
function<int(int)>
s in the latter two cases... ?
-
@cvi said in C++: std::function, implicit conversions, ambiguous overloads, and a bit of a mess:
I think original approach would result in an inner
function<int(int)>
s in the latter two cases... ?Only for this one (by necessity)
// simplifying fun wrapper, SimpleFun -> Fun FunctionWrapper(const SimpleFun<T>& f) : fun([=](T x, bool) { return f(x); }) {}
but not for this one, I think:
// no-op wrapper, Fun -> Fun FunctionWrapper(const Fun<T>& f) : fun(f) {}
Here, it creates a std::function to wrap the callable before being passed as an argument, but then the argument only gets copied.
I'll figure something out. It will probably have no performance impact at all, but if it does I can maybe add more SFINAE overloads to your code. Most likely, I'll just ignore it. ;)
-
@topspin said in C++: std::function, implicit conversions, ambiguous overloads, and a bit of a mess:
Here, it creates a std::function to wrap the callable before being passed as an argument, but then the argument only gets copied.
Yeah, but my version does the same:
template< class tCallable > FunctionWrapper( tCallable&& aCallable, std::enable_if_t< is_callable<tCallable,T,bool>::value, int > = 0 ) : fun( std::forward<tCallable>(aCallable) ) {}
I mean, there's some additional line noise thanks to the perfect forwarding ... but if you say
std::function<int(int, bool)> f; FunctionWrapper<int> w(f);
then
tCallable
will deduce tostd::function<int(int,bool)>&
and the whole thing becomes the same as your constructor. (Iff
wereconst
,tCallable
would become astd::function<int(int,bool>) const&
instead.)
-
@cvi said in C++: std::function, implicit conversions, ambiguous overloads, and a bit of a mess:
Yeah, but my version does the same
I have simply overlooked that. Just saw one constructor doing a capture and assumed the other would, too.
So it's only doing work that's strictly necessary. Perfect.