Since the current Internet provides beat-effort service, timely delivery of data is not guaranteed for video streaming applications. Providing Quality of Service(QoS) in such a setting requires server/client to be network-aware and adaptive. We present a dynamic rate and quality adaptation scheme where the server varies its sending rate (without varying the quality) to adapt to the network and client conditions and only as a last resort, does quality adaptation. Our approach is unique because the server's sending rate is calculated based on the client's varying demand and the network status. We do not model the network as a black-box but instead augment end-point observations with feedback from the network and hence represent its status more precisely. We also propose a method to reduce the amount of control information needed to make adaptation decisions.