you can actually do the broadcast explicitly in the dataframe api — dfA.join(broadcast(dfB), …). If you are sure that the dfB will always be small, thats a safe bet and a good practice

you may also want to think about switching order of the operations in the pipeline (ie, first agg and then join) — assuming the logic of the aggregation allows it, of course. Or maaaybe even split it into three: first do aggregations that do not require the join, then do the join, and lastly the final aggregation. The logic behind this is the actions that reduce the total volume of the data (filters, aggregations), are best done as early as possible