To configure the Widgets, please go to Dashboard > Appearance > Customize > Widgets.
In the widgets panel, you can find different widget sections whose controls will help to customize the dedicated widgets as shown in the image.
For example, the Woocommerce Right Sidebar widget shown below.
