thank you for taking the time to write this. I couldn't move the communicator call outside the transaction because then I'd have to also move the first update operation in the transaction too because the communicator depends on it:
ActiveRecord::Base.transactiondorecord1.update!(some_data)# want to rollback thisexternal_service_response=Communicator.call!(record1,save_error: true)# dependes on the previous line to make the callrecord2.update!(external_service_response)end
it's not great design but it's what i'm working with.
Regarding your concern about performance, I'm not actually spawning N + 1 connections; there's a fixed number of connections in the pool that you can set (defaults to 5), and ActiveRecord::Base.connection_pool handles the case when there are more threads than connections as per the docmentation:
It will also handle cases in which there are more threads than connections: if all connections have been checked out, and a thread tries to checkout a connection anyway, then ConnectionPool will wait until some other thread has checked in a connection.
so, for example, if we set the max connections in the pool to 5 and we have 10 threads, 5 connections will be used by 5 threads and when one of these threads finishes using a connection, it will be checked in to the pool and ready to be used by one of the other 5 threads and so on.
I really like the idea of savepoints. I'd even prefer it over my current solution, but I'm not sure how to make it work in this case because the statement I don't want to roll back is sandwhiched between two statements I want to roll back :D Thank you for telling me about this, though!
Regarding your concern about performance, I'm not actually spawning N + 1 connections; there's a fixed number of connections in the pool that you can set (defaults to 5), and ActiveRecord::Base.connection_pool handles the case when there are more threads than connections as per the docmentation
That's true, I don't know why but I understood that you said Rails was creating a new pool for each thread 😅
I really like the idea of savepoints. I'd even prefer it over my current solution, but I'm not sure how to make it work in this case because the statement I don't want to roll back is sandwhiched between two statements I want to roll back :D Thank you for telling me about this, though!
What about moving using an explicit transaction only for the first two statements? Let's see:
if update #1 breaks, you catch the error, the write wasn't done anyway, the call to the external service won't be made and you're fine
if the call to the external service returns an error response you explicitly rollback the transaction, the first update gets undone, but you still have the error response
update #2 checks if the error response is populated, if so it writes on disk, otherwise it just doesn't do anything.
I don't understand what you really want to accomplish then :D
The logic seems to be too complicated for these three lines of code.
Update 1 should be rolled back if there's an error in Communicator but the save inside communicator should happen regardless of there being an error and update 2 should happen only if communicator went along fine?
If you have control on the Communicator object you should probably keep the logic of calling the external service and remove the one that's giving you pain (which shouldn't be there in theory) which is the part about saving the error in the DB.
If the communicator only communicates you can then leave all the DB logic in your "main transaction" and decide what to do. Something like:
# update record 1
# call the service with record 1
# did the call went fine? yes? move on
# did the call raise an error? store the error in a variable
##### rollback the transaction
##### save the error outside the transaction bloc
Update 1 should be rolled back if there's an error in Communicator but the save inside communicator should happen regardless of there being an error and update 2 should happen only if communicator went along fine?
yes, exactly.
If you have control on the Communicator object you should probably keep the logic of calling the external service and remove the one that's giving you pain (which shouldn't be there in theory) which is the part about saving the error in the DB.
it is required to save the error response to the db, but maybe it could be done outside of communicator. I'll have to think about this some more.
thank you again, rhymes, for taking the time to discuss this!
it is required to save the error response to the db, but maybe it could be done outside of communicator. I'll have to think about this some more.
See it like this. The service object should be responsible for one thing only. In the future you might want to do N different things with the response. You might want to save it on the DB, you might want to log it on an external service, you might want to extract info from it. Separating the two steps should simplify the logic. In an extreme case you might even save the error later and asynchronously ;)
Hey rhymes,
thank you for taking the time to write this. I couldn't move the communicator call outside the transaction because then I'd have to also move the first update operation in the transaction too because the communicator depends on it:
it's not great design but it's what i'm working with.
Regarding your concern about performance, I'm not actually spawning N + 1 connections; there's a fixed number of connections in the pool that you can set (defaults to 5), and
ActiveRecord::Base.connection_pool
handles the case when there are more threads than connections as per the docmentation:so, for example, if we set the max connections in the pool to 5 and we have 10 threads, 5 connections will be used by 5 threads and when one of these threads finishes using a connection, it will be checked in to the pool and ready to be used by one of the other 5 threads and so on.
I really like the idea of savepoints. I'd even prefer it over my current solution, but I'm not sure how to make it work in this case because the statement I don't want to roll back is sandwhiched between two statements I want to roll back :D Thank you for telling me about this, though!
That's true, I don't know why but I understood that you said Rails was creating a new pool for each thread 😅
What about moving using an explicit transaction only for the first two statements? Let's see:
Something like:
would that do?
hmm, good thinking, but I don't think it would work :D the save error operation inside the communicator class would still be rolled back.
What do you think?
I don't understand what you really want to accomplish then :D
The logic seems to be too complicated for these three lines of code.
Update 1 should be rolled back if there's an error in
Communicator
but the save inside communicator should happen regardless of there being an error and update 2 should happen only if communicator went along fine?If you have control on the
Communicator
object you should probably keep the logic of calling the external service and remove the one that's giving you pain (which shouldn't be there in theory) which is the part about saving the error in the DB.If the communicator only communicates you can then leave all the DB logic in your "main transaction" and decide what to do. Something like:
Does it make sense?
yes, exactly.
it is required to save the error response to the db, but maybe it could be done outside of communicator. I'll have to think about this some more.
thank you again, rhymes, for taking the time to discuss this!
See it like this. The service object should be responsible for one thing only. In the future you might want to do N different things with the response. You might want to save it on the DB, you might want to log it on an external service, you might want to extract info from it. Separating the two steps should simplify the logic. In an extreme case you might even save the error later and asynchronously ;)
yeah, maybe another service could handle dealing with the response and whatever we want to do with it :D