In this paper, we present an approach combining both region selection and user point selection for user-assisted segmentation as either an enclosed object or an open curve, investigate the method of image segmentation in specific medical applications (user-assisted segmentation of the media-adventitia border in intravascular ultrasound images, and lumen border in optical coherence tomography images), and then demonstrate the method with generic images to show how it could be utilized in other types of medical image and is not limited to the applications described. The proposed method combines point-based soft constraint on object boundary and stroke-based regional constraint. The user points act as attraction points and are treated as soft constraints rather than hard constraints that the segmented boundary has to pass through. The user can also use strokes to specify region of interest. The probabilities of region of interest for each pixel are then calculated, and their discontinuity is used to indicate object boundary. The combinations of different types of user constraints and image features allow flexible and robust segmentation, which is formulated as an energy minimization problem on a multilayered graph and is solved using a shortest path search algorithm. We show that this combinatorial approach allows efficient and effective interactive segmentation, which can be used with both open and closed curves to segment a variety of images in different ways. The proposed method is demonstrated in the two medical applications, that is, intravascular ultrasound and optical coherence tomography images, where image artefacts such as acoustic shadow and calcification are commonplace and thus user guidance is desirable. We carried out both qualitative and quantitative analysis of the results for the medical data; comparing the proposed method against a number of interactive segmentation techniques.