You will soon have the option to have a less chatty Google Assistant on your phone. Google just updated its support site that shows how you can turn off Assistant speech output on your phone. But since the support site was only recently updated, it isn't immediately available. You're going to have to keep checking back to see if the feature comes to you.
To turn off speech output, you need to open Assistant and tap the explore icon. Open the settings, go to the Assistant tab, and tap Phone. Under Voice and speech, there will be an option to change speech output to Hands-free only. Once you've enabled this feature, Assistant will still do what you say, but it won't speak out loud to tell you what it's doing. You will still be able to read the responses on the screen.
Source: Android Police